MS Source Server - source stream is apparently not there when viewing with srctool

Question

Hi,

I have been playing around with the MS Source Server stuff in the MS Debugging Tools install.

At present, I am running my code/pdbs through the Subversion indexing command, which is now running as expected. It creates the stream for a given pdb file and writes it to the pdb file.

However when I use that DLL and associated pdb in visual studio 2008, it says the source code cannot be retrieved.

If I check the pdb against srctool is says none of the source files contained are indexed, which is very strange as the process prior ran fine.

If I check the stream that was generated from the svnindex.cmd run for the pdb, srctool says all source files are indexed.

Why would there be a difference?

I have opened the pdb file in a text editor and I can see the original references to the source files on my machine (also under the srcsrv header name) and the new "injected" source server links to my subversion repository).

Should both references still exist in the pdb? I would have expected one to be removed?

Either way, visual studio 2008 will not pick up my source references so I am a bit lost as to what to try next. As far as I can tell, I have done everything I should have.

Anyone have similar experiences?

Many thanks.

Answer 1

There is an option in the Options\Debugging\Symbols of Visual Studio to emit logs of the source server.
Also you need the latest version of srcsrv.dll, just download the latest WinDBG copy the DLL from there and make sure that Visual Studio uses it.

Answer 2

I have resolved my problem - the source file path that was written to the PDB during build was slightly different to the one written as part of the Subversion source index task.

This must invalidate the source search within Visual Studio as the two paths don't match.

Also writing my own simplified source index stream to my PDB files from a custom NAnt task, which hooks up to Vault, our SCM system.

Answer 3

Not been on here in ages, sorry. It's quite specific to our company's needs. Should be enough to show what is going on though.

Code snippets below:

PdbFile.cs

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;

namespace Code.Integration.SourceIndex
{
    public class PdbFile
    {
        private FileInfo _pdbFile;

        public FileInfo Pdb
        {
            get { return _pdbFile; }
        }

        public PdbFile(FileInfo pdbFile)
        {
            if (pdbFile == null)
                throw new ArgumentNullException("pdbFile");

            if (!pdbFile.Exists)
                throw new ArgumentException(string.Format("Pdb file specified \"{0}\" does not exist.", pdbFile.FullName), "pdbFile");

            _pdbFile = pdbFile;
        }

        // Read source files
        // Write source stream
    }
}

PdbUtil.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Text;

namespace Code.Integration.SourceIndex
{
    public static class PdbUtil
    {
        private static readonly string SRCTOOL_PATH_1 = @"C:\Program Files\Debugging Tools for Windows\srcsrv\srctool.exe";
        private static readonly string SRCTOOL_PATH_2 = @"C:\Program Files\Debugging Tools for Windows (x86)\srcsrv\srctool.exe";
        private static readonly string PDBSTR_PATH_1 = @"C:\Program Files\Debugging Tools for Windows\srcsrv\pdbstr.exe";
        private static readonly string PDBSTR_PATH_2 = @"C:\Program Files\Debugging Tools for Windows (x86)\srcsrv\pdbstr.exe";

        private static string SRCTOOL = "";
        private static string PDBSTR = "";

        static PdbUtil()
        {
            if (File.Exists(SRCTOOL_PATH_1))
                SRCTOOL = SRCTOOL_PATH_1;
            else if (File.Exists(SRCTOOL_PATH_2))
                SRCTOOL = SRCTOOL_PATH_2;

            if (File.Exists(PDBSTR_PATH_1))
                PDBSTR = PDBSTR_PATH_1;
            else if (File.Exists(PDBSTR_PATH_2))
                PDBSTR = PDBSTR_PATH_2;
        }

        private static void EnsureToolsExist()
        {
            if (string.IsNullOrEmpty(SRCTOOL))
                throw new ApplicationException(string.Format("SRCTOOL does not exist. Is it installed?", SRCTOOL));

            if (string.IsNullOrEmpty(PDBSTR))
                throw new ApplicationException(string.Format("PDBSTR does not exist. Is it installed?", PDBSTR));
        }

        public static List<string> ReadSourceFiles(PdbFile pdb)
        {
            EnsureToolsExist();

            ProcessStartInfo info = new ProcessStartInfo(SRCTOOL);

            info.UseShellExecute = false;
            info.RedirectStandardError = true;
            info.RedirectStandardOutput = true;
            info.Arguments = string.Format("-r \"{0}\"", pdb.Pdb.FullName);

            string output;
            string errors;

            using (Process p = Process.Start(info))
            {
                output = p.StandardOutput.ReadToEnd();
                errors = p.StandardError.ReadToEnd();

                p.WaitForExit();
            }

            if (!string.IsNullOrEmpty(errors))
                throw new ApplicationException(string.Format("Error reading pdb source files \"{0}\".", errors));

            List<string> result = new List<string>();

            if (!string.IsNullOrEmpty(output))
            {
                foreach (string item in output.Split('\r', '\n'))
                {
                    string sourceFile = item.Trim();

                    if (string.IsNullOrEmpty(sourceFile))
                        continue;

                    result.Add(sourceFile);
                }
            }

            return result;
        }

        public static void WriteSourceFileStream(PdbFile pdb, FileInfo stream)
        {
            EnsureToolsExist();

            ProcessStartInfo info = new ProcessStartInfo(PDBSTR);

            info.UseShellExecute = false;
            info.RedirectStandardError = true;
            info.RedirectStandardOutput = true;
            info.Arguments = string.Format("-w -s:srcsrv -p:\"{0}\" -i:\"{1}\"", pdb.Pdb.FullName, stream.FullName);

            string output;
            string errors;

            using (Process p = Process.Start(info))
            {
                output = p.StandardOutput.ReadToEnd();
                errors = p.StandardError.ReadToEnd();

                p.WaitForExit();
            }

            if (!string.IsNullOrEmpty(errors))
                throw new ApplicationException(string.Format("Error writing to pdb \"{0}\".", errors));
        }
    }
}

SourceIndexTask.cs

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;

using NAnt.Core;
using NAnt.Core.Attributes;
using NAnt.Core.Types;

using Code.Integration.SourceIndex;

namespace Code.Integration.NAnt.Tasks.SourceIndex
{
    [TaskName("codesourceindex")]
    public class SourceIndexTask : Task
    {
        private FileSet _pdbs = new FileSet();
        private string _build = "0.0.0.0";
        private string _repositoryName = "";
        private string _repositoryProjectStart = "";
        private string _user = "";
        private string _password = "";
        private string _server = "";

        #region Properties

        /// <summary>
        /// FileSet of pdbs to process.
        /// </summary>
        [BuildElement("pdbs")]
        public FileSet Pdbs
        {
            get { return _pdbs; }
            set { _pdbs = value; }
        }

        /// <summary>
        /// Build label to extract.
        /// Default is "0.0.0.0".
        /// </summary>
        [TaskAttribute("build", Required = true)]
        [StringValidator(AllowEmpty = false)]
        public string Build
        {
            get { return _build; }
            set { _build = value; }
        }

        /// <summary>
        /// Name of repository we are working on.
        /// </summary>
        [TaskAttribute("reponame", Required = true)]
        [StringValidator(AllowEmpty = false)]
        public string RepositoryName
        {
            get { return _repositoryName; }
            set { _repositoryName = value; }
        }

        /// <summary>
        /// Name of start folder within project we are working on. i.e. if the 
        /// repository was "Code 2-0" then the repository project start could be "/Code/Trunk"
        /// or "/Code/Branches/1.0.0.123/"
        /// </summary>
        [TaskAttribute("repoprojectstart", Required = true)]
        [StringValidator(AllowEmpty = false)]
        public string RepositoryProjectStart
        {
            get { return _repositoryProjectStart; }
            set { _repositoryProjectStart = value; }
        }

        /// <summary>
        /// Vault user with repository access.
        /// </summary>
        [TaskAttribute("user", Required = true)]
        [StringValidator(AllowEmpty = false)]
        public string User
        {
            get { return _user; }
            set { _user = value; }
        }

        /// <summary>
        /// Vault user password.
        /// </summary>
        [TaskAttribute("password", Required = true)]
        [StringValidator(AllowEmpty = false)]
        public string Password
        {
            get { return _password; }
            set { _password = value; }
        }

        /// <summary>
        /// Location of Vault server.
        /// </summary>
        [TaskAttribute("server", Required = true)]
        [StringValidator(AllowEmpty = false)]
        public string Server
        {
            get { return _server; }
            set { _server = value; }
        }

        #endregion

        protected override void ExecuteTask()
        {
            try
            {
                WriteFiles();
            }
            catch (Exception exception)
            {
                throw new BuildException("Source indexing could not be completed.", Location, exception);
            }
        }

        private void WriteFiles()
        {
            foreach (string fileName in Pdbs.FileNames)
            {
                Log(Level.Info, string.Format("Processing '{0}'.", fileName));

                PdbFile pdb = new PdbFile(new FileInfo(fileName));
                List<string> sourceFiles = PdbUtil.ReadSourceFiles(pdb);
                string tmpFile = Path.GetFullPath(Path.GetTempFileName());

                try
                {
                    using (StreamWriter sw = new StreamWriter(tmpFile))
                    {
                        sw.WriteLine("SRCSRV: ini ------------------------------------------------");
                        sw.WriteLine("VERSION=1");
                        sw.WriteLine("VERCTRL=VAULT");
                        sw.WriteLine("DATETIME=" + DateTime.Now.ToUniversalTime().ToString("u"));
                        sw.WriteLine("SRCSRV: variables ------------------------------------------");
                        sw.WriteLine("VAULT_USER=" + User);
                        sw.WriteLine("VAULT_PASS=" + Password);
                        sw.WriteLine("VAULT_SRV=" + Server);
                        sw.WriteLine("VAULT_EXTRACT_TARGET=%targ%%fnbksl%(%var3%)\\%var4%\\%fnfile%(%var1%)");
                        sw.WriteLine("VAULT_EXTRACT_FOLDER=%targ%%fnbksl%(%var3%)\\%var4%");
                        sw.WriteLine("VAULT_EXTRACT_CMD=\"C:\\Program Files\\SourceGear\\Vault Client\\vault.exe\" getlabel -host %vault_srv% -user %vault_user% -password %vault_pass% -repository \"%var2%\" \"$%var3%\" %var4% -nonworkingfolder \"%vault_extract_folder%\" > \"%vault_extract_target%.log\"");
                        sw.WriteLine("SRCSRVTRG=%vault_extract_target%");
                        sw.WriteLine("SRCSRVCMD=%vault_extract_cmd%");
                        sw.WriteLine("SRCSRV: source files ---------------------------------------");

                        foreach (string sourceFile in sourceFiles)
                        {
                            // Will build against something like:
                            // D:\CruiseControl.NET.Working\Solutions 2.0\Code\Code\Trunk\Working\Solution\..
                            // Don't want "Working" folder name in there either.
                            // Need to generate Vault repo path to asset:
                            // /Code/Trunk/Solution/..

                            // 1. Pass in repo start - /Code/Trunk
                            // 2. Redirect slashes and search for \Code\Trunk
                            // 3. Find first index and split at index to get - \Code\Trunk\Working\Solution\..
                            // 4. Remove "Working\"
                            // 5. Flip slashes again to get - /Code/Trunk/Solution/..

                            // Problems:
                            // 1. Passing in "Trunk" - would need to work that out dynamically over time

                            string repositoryPath = sourceFile;

                            int index = sourceFile.IndexOf(RepositoryProjectStart.Replace("/", @"\"));
                            if (index != -1)
                                repositoryPath = sourceFile.Substring(index);

                            repositoryPath = repositoryPath.Replace(@"Working\", "");
                            repositoryPath = repositoryPath.Replace(@"\", "/");

                            sw.Write(sourceFile);
                            sw.Write("*");
                            sw.Write(RepositoryName);
                            sw.Write("*");
                            sw.Write(repositoryPath);
                            sw.Write("*");
                            sw.Write(Build);
                            sw.WriteLine();
                        }

                        sw.WriteLine("SRCSRV: end ------------------------------------------------");
                    }

                    Log(Level.Debug, string.Format("Generated stream '{0}'.", File.ReadAllText(tmpFile)));

                    // Write the stream to the pdb file
                    PdbUtil.WriteSourceFileStream(pdb, new FileInfo(tmpFile));

                    Log(Level.Info, "Written stream to pdb.");
                }
                finally
                {
                    if (File.Exists(tmpFile))
                        File.Delete(tmpFile);
                }
            }
        }
    }
}