One of the things I really love about programming is the tradition of building your own tools. Sometimes it's out of necessity. What you need doesn't exist. And sometimes it's out of ignorance (I hand-wrote many parsers before learning about lex & yacc).
What programming tools have you built for yourself?
Waaaay back when I was first starting uni, they were teaching us Java, and I got sick of having to enter in all the commands to compile and run our programs, so I wrote an incredibly simple IDE using (wait for it), QBasic.
A small HTML editor that highlight HTML keyword. This is back in the early 2000, now I use for small editor NotePad++
The first tool I wrote of any substance was a 6502 machine code disassembler. Like a lot of on-the-side projects, the tool quickly became the main on-the-side, and whatever prompted me to write it is lost in the attic of my memory.
I was probably 14, on a BBC B, writing the majority of it in BASIC, with some bits in assembler. I could bore you with the extra cool features (e.g. 2 passes for labelled jumps) but that'd get this answer into nostalgia.
I once wrote a javadoc like utility for maintaining documented database schemas with SQL create table statements and specially formatted comments on the table and column definitions. It parsed the SQL file and generated a LaTeX or MIF data dictionary document.
Also, I've done quite a few code generators of one sort or another.
My first ever C program was a hex dump utility.
Extended Clipboard handler that worked over the local network. (Both text & files) Was handy back when TS couldnt handle files over clipboard.
Visual Studio Add-in that let me configure some XML files for the "in-house" multi-langage system. So in VS when I select a string and do right click and "Convert to language system" I have a form that popup and let me write the string in multiple language. After, all is saved... when I compile I see the string that was saved in the XML files. This is the tool I use every day.
I've written a number of specialized code generators (usually outputting C++). These have tended to be project-specific and have saved me more coding than I can shake a stick at.
The two that I used the most were my .NET Class Generator application which generated information object classes from .xls input, saved a bunch of time in C# before Auto Properties.
Also the most popular of all is one that I made available to the public and that is a Regular Expression Tester that supports groups and other items, it has made my work with RegEx a breeze!
I used to write my own unit testing frameworks back when dinosaurs roamed the Earth. This was, of course, before JUnit and TDD, and even before we called it unit testing. I'd written frameworks for testing functions in C, C++, QBasic, and LabVIEW before going back to school to get my CS degree and seeing all the really cool testing tools that we have now.
Without wanting to spam/adervtise, but since you ask: All of the ones listed on my site were initially built for myself.
They are used in my buildscripts because I did not investigate on doing "proper" MSBuild tasks yet (did not want to spend time with that just yet).
Since I manage a Source Control Management (ClearCase), and I had to be able to integrate its various commands in a in-house release management tool developed in java.
For that, I developed (back in 2002) an encapsulation of System.Runtime in order to:
Example code here (that one does not log anything)
The ability to literally follow what is going on when a java program execute a ClearCase command has been incredibly helpful. Even the more recent Java6 Console does not achieve quite what I do with this 'Shell' class.
For reasons I can't remember anymore, the toolchain I was using to target an embedded system couldn't be scripted using batch (this was on Win95). I needed a build script, so I created my own "script" in C++ that invoked the various tools and then initiated the download to the embedded device.
Also, when I was in highschool, I had permission to use gcc on one of the school computers so that I could work on random projects during my down time, but they had the system locked down so that copying executables to and from floppy disks was impossible using the cp command (perhaps to combat the spread of viruses?). I coded myself a new copy command with the compiler as my first project :)
Neither of those are very exciting, I'm afraid. I try as much as possible to not reinvent the wheel.
A jar dependency-hell visualizer, to fix an extremely bloated ant script. Made me use the wonderful ASM library.
I like to write code generators for the mundane parts. For example, if you have to write .NET DependencyProperties, the code is verbose, repetitive, and tedious. A code generator is just the ticket.
I usually use PowerShell for external DSLs.
I know this is a little sad, but I always have an Excel Macro ready to paste bugs from Bugzilla and format them as tightly as possible to get all my bugs on a single double-sided piece of 8.5x11.
Bug tracking software never seems to have a way to compress and list your current tasks on paper.
I wrote some code generators to avoid repetiting tasks.
I also wrote a metric and program structure tool. To analyse legacy software.
Recently, I have written a registration key system for distribution of commercial applications. Currently am fleshing out how to inject the dynamically generated code to handle the validation into an existing assembly. The primary interface is through a Visual Studio add-in that allows you to select the algorithm, define a custom algorithm, and apply it to a specific assembly.
In the past, I have written more data conversion utilities and scubbers than I care to remember, and countless refactoring macros that have (thankfully) been replaced by ReSharper.
i recently got tired of downloading, processing and presenting statistics from the company websites for the guys in marketing. I wrote this tool which they could install on their laptops. From there they could handle the whole process at the click of only one button. That's all the tool has, one button!!! How user friendly could you get for the guys down at marketing.
I wrote a little applet that sits in my system tray to log configuration changes whenever I install, update or uninstall software, change the registry or tweak the system.
I wrote my own Windows Grep tool, which I have used almost daily for a year or more...it is a very satisfying feeling! For some reason it seems to be much faster than the other tools I have tried, probably through liberal use of memory :)
A Perl script that scrubs CVS log, presents check-ins grouped by comment and by bug database entry, and (the most important thing) allows me to generate cvs update -j commands to merge changes on per-bug basis.
Various framework for writing system tests, specifically for FIX protocol exchange trading systems and for XWindows GUI. All that in Perl.
Back in the dark days, I wrote a number of addins for the Brief editor using their pseudo-Lisp macro language. Later when I move to PWB (Microsoft's first multi-langauge IDE), I wrote a couple addins for it -- they were in compiled C code, which was kinda impressive, since this was in MSDOS back in the days before we had DLLs.
One, which I'd written for both Brief & PWB, would take a highlighted block of text and sort the lines. I've tried translating that into a VS Addin, but when looking at a selected block now, you lose the line & column coordinates.
LOTS (and thank you for the question ;) )
I guess it all started when I tried to write a LISP interpreter back in the BASIC times, with list cells being stored in DIM'ed arrays. And I always wanted to write a file format analyzer, but never found the time to do it.
From database that one of my projects used I derived a documentation database and a Word macro to create some reference documents, and a couple of ASP pages to display this information. These were mainly project-specific hacks.
Before ASP.Net came along, I developed an XML-based generic list-and-forms framework in ASP. Table information and list and form definitions where stored in XML. The ASP application used XSLT to generate ASP pages containing the data on the fly.
Forms Toolkit was written to search and manipulate Oracle Forms files using the Oracle Forms API.
Recently I try to be less project-specific, and try to share my dev tools: I wrote a utility to check for XHTML compliance, search and extract data from aspx pages (graspx), and a tool to script MSSQL database objects (SMOscript) similar to the sql2000 tool scptxfr.
The latest software I am working on is dbscript, which supports importing and uploading of database scripts, definition of value scripts for TSQL and C#, schema comparison, XML generation. It's sufficient for my needs, but a still work-in-progress for public use.
I describe and document these tools on my blog, downloads are available from my homepage.
ASP.NET ViewState Helper to help me troubleshoot issues with ViewState, and just to see what is being stored in there.
I wrote a command line tool to find and clear out subversion conflicts. (With user input)
Way back in the dark ages, I wrote a tool that would generate callee/caller info (e.g. info necessary for a call graph) for VB6.
Regex tool: I found myself writing code which required a lot of regex search and replaces, and I didn't want to learn Vim or pay for RegexBuddy, so I wrote my own tool and use it everyday.
Source code line counter: a little tool which counts the number of lines of code in a folder. Has lots of different rules for excluding arbitrary lines in different languages.
Other projects include:
Stock predicting tool: I was really into tracking the stock market, so I wrote a little program which would download data from the internet and perform a lot of technical analysis on the data and forecast bulls and bears. My program could choose stocks with an impressive 60% accuracy ("accurate" prediction means that, within one month of the prediction, a bull increases in value by 15% and bear loses 10% value).
Redneck speak: I owned a somewhat controversial political website. A rival website used to harass my users, so I wrote a little page that converted the rival website's content into Redneck grammar.
Grocery list: contained a list of foods + prices, contained a list of recipes. Program allowed me to pick from a list of foods or pick a recipe, and it would automatically import all of the foods and proper quantities into a list and estimate the total cost. This program is used everytime I go shopping, and is frequently accurate within a few dollars.
Cat fancier: screensaver I wrote for my cats. Has two modes: 1) Serene, which displays a scene of a window overlooking a street. 2) Insanity, which displays highspeed laser dots and flapping butterflies to drive my cats nuts.
Keylogger: my sister wanted to spy into her husbands email, so I wrote a keylogger for her to capture passwords.
Playing with XNA: I was fascinated with Microsoft XNA, so I bought a few books on the topic. My pride and job was a program that allowed me to plug in an XBox controller, fly a spaceship around in cornflower blue space, and shoot missles. You could dynamically load any model in place of the spaceship, which prompted some funny conversations, my favorite being "why is that frog flying around and shooting missles?"
Message board statistics: I'm a messageboard junkie, so I wrote a program that would go through a database and crunch a bunch of meaningless statistics about users, such as their frequency of posts they write on different days of the week and times of day, average length of sentence, average length of post, etc.
There are lots of other miscellaneous projects I could add to that list, but I think if I say too much some of you will rightfully conclude that I have no life.
Pretty much anything I do that's repetitive I'll try to write a tool to speed up the process if I have a chance to. We were recently rewriting part of our payroll system. I wrote a calculator that allowed adding and subtracting hours and minutes so I could check the reports it generated. There's also the program for quickly looking up SQL codes. And one for searching PowerBuilder pbls to find which one contains a given object. I wrote one tool that's basically just a textbox with the always on top attribute set to true. And a program to insert comments into a PowerBuilder window formatted according to company standards.
So many things. Actually, our tool-building instinct is something we have to look at carefully. I'll just come right out and say this: I would rather build tools than build applications.
I think most of us have a little bit of that. I have heard it described as a recursive problem that can really eat a lot of project time.
To answer the question, my biggest tool: I read Rocky Lhotka's book on CSLA and built my own ORM tool.
That took awhile, but I believe it took me to a different level as a developer. We still use it here at Inntec, although the day is coming when it will be replaced. There are some awesome ORM tools now.
I built a tool that lets me tag important information in method documentations (e.g., don't call this from the UI thread), and then highlights the calls to methods that have those tags so I get some advanced warning that I should probably look at the docs.
I also built a tool that tracks everything I do in Eclipse and lets me quickly record notes to generate and present a journal that I can browse.
I wrote an assembler IDE in C# a couple of weeks ago. I would still use it - but I mucked up the regexes and once you get to abou 2 lines, it takes a minute to do the syntax highlighting!
Sting thisIsIt = " I've build a \"quotator\" that transforms regular strings into java-code strings \n"
+ " and the other way around. Very handy for sql tests \n"
+ " \n"
+ " For instance: \n"
+ " \n"
+ " select * from dual \n"
+ " where 1 = 0 \n"
+ " \n"
+ " is trasformed to: \n"
+ " \n"
+ " " select * from dual " \n"
+ " + " where 1 = 0 \n" \n"
+ " \n"
+ " This is very easy to do in any decent Ide ( and even IntelliJ Idea has it built-in ) \n"
+ " But the good thing here is that it takes and leaves the data in the clipboard so I just have to: \n"
+ " \n"
+ " ctrl-C double-click-on-my-jar ctrl-v \n"
+ " "
+ " \n"
+ " :) \n"
For Visual Studio, A small c++ class wizard that uses modifiable templates .h/.cpp files . A macro that creates function definitions in the cpp for the highlighted declarations in the corresponding header .
The most valuable: (circa 1994) a VBA macro that would parse Microsoft Word 2.0 document to extract translation tables for our data warehouse. This gave the business users complete responsibility for maintaining these tables, with commentary, in a form that could be printed and passed around.
The most useful: (circa 1986) a minimal debugger for MS-DOS, that could be linked into a device driver and therefore available as soon as the machine started. Supported breakpoints and arbitrary peek/poke, but not disassembly (hey, if you were good you could do that from a hex dump :-). Allowed me to debug the printing hooks for one of the first PC faxboards.
A Lua implementation of Perl's Data::Dumper. It makes debugging data structures breeze.
I've once written a program to generate about one or two thousand lines of repetitive code. I later realized I could have used the Method class.
I've written a few code generators. If I do something more than twice, I look to automate it.
Various "diff" utilities. They weren't always available.
YAPA (Yet Another Performance Analyzer) that automated call-stack sampling and displayed the results in a butterfly view. Went obsolete with DOS.
Dynamic Dialogs little horn-tooting here, a technique that is so simple and nifty, but also so hard to explain, that I've practically spoiled myself for doing UIs any other way.
Lots of parsers, compilers, code generation hacks of different sorts.
Dumb theorem provers. Used to prove that bubble-sort works. woopee.
Spelling correction algorithm based on trie-search.
FFT algorithm in run-coded images.
Most recently, Halp makes writing small programs noticeably faster and more fun for me. It's a little like having a spreadsheet for code embedded in my Emacs source-code buffer. I've noticed that I write programs now that I wouldn't have bothered to before, from trying things out and assembling them into a program with less friction from switching between coding and testing, etc.
Not an add in but where I work we have a standardized way of performing database calls with stored procedures. Well ... I got sick of writing basically teh same thing over and over with just different parameters etc. So in my off time I wrote a program that just allows me to choose a stored procedure and click "generate" and it spits out a chunk of code that I can just copy and paste into an application. It has saved me untold hours of rewriting methods with different parameters. Made my life alot easier.
I created an Ant task to compile a Visual Basic 6 project. Some features of the VB6 compiler are not accessible from the command line, so I had to programmatically alter some of the project's properties.
The following properties can be set in the Ant script which are set either through the command line or through temporarily changing the project's source.
(OK, this sounds pretty useless, but if you have to stick with VB6 you have to think of something to make your coding life easier)
Gobs and gobs. A few notables:
package.xml changelog from SVN commit messagespackage.xmlphp -l (lint/syntax check)Because LANSA source (RDML) is stored in a database on the iSeries with one or more records per line of source (1 for every 80 characters), and LANSA has no built-in version control (just a crude check-in, check-out with no diff/merge/roll-back), I built a windows app in .Net that exports source to text files and allows me to launch diffmerge or check-in/refresh from a subversion server.
I wrote
I wrote my own code snippet tool, aptly named "Snip-a-Dillie-O". While I enjoyed the Visual Studio snippet manager, I found it a little tedious to get up and running and maintain. I also had a variety of SQL snippets, commenting, and miscellaneous type things that I use frequently and couldn't put in there. I also wanted something to try out C# and SQLite with on the side. I have a "favorites" tab for it, drag and drop capabilities, and the standard copy paste type stuff. I need to retool it so I can minimize to the system try and access the favorites that way.
I have a web based app that will parse Log4Net logs generated in their default XML format called HackSaw (in honor of the original ChainSaw) that I use to parse my log files. Nothing fancy, but it gets the job done.
I also have a "Formatted Text Loader" library that uses an XML "layout file" and will parse a given text file into a strongly typed DataTable. This became invaluable to my at my previous job since there were lots of legacy log/data files there that I need to process. I even set it up so that if you can write a regular expression with groups that captures all your data, you'll get a table out of that.
Oh, and thanks to the SO folks here, I have a small tool that will decode those ASCII Hex Encoded SQL Injection attacks that may be hitting your server thanks to the ASPRox botnet virus that comes and goes. If you're interested in what code they're trying to execute, this'll show ya without having to decode it directly on your SQL Server.
I know I'm pulling out the golden shovel to describe these, but they're nothing to fancy. Hopefully somebody will make them look better unless I get more free time to code 8^D
I've written a tiny command line app to write my file header comments for C# and C++ source files, because I needed to write the same comment based on the filename and the date created and of course the author name which the user will specify, so my tool generated that for me because I've had the existing files and I just needed to put that on the top :)
We use ant and gnat for some big projects, and the build tool dumps lots of "building, N% complete" messages.
I wrote an output filter that identifies these lines, jumps the cursor back one line so it is overwritten by the next line (instead of scrolling), and sets my "screen" window title to include the ant task and % complete, so I can see how the build is coming while I'm working on another window.
My pet is the Umbraco Interaction Layer which is a .NET class generator for Umbraco Document Types. It acts as an API replacement when you want to programatically interact with Umbraco
I hated the interface to yacc so much that I built my own parser generator, which generates LL(1) and SLR(1) parsers in three languages. I don't have to declare keywords and I can use EBNF notation.
The project never got to the point where I was willing to let other people use it, and the professionals in Holland make far better parser generators than I can hope to emulate, but I learned an enormous amount by going back to especially the tutorial paper by Aho and Johnson and implementing what I found there.
I wrote NValidate (at www.nvalidate.org), so I could replace this:
if(string.IsNullOrEmpty("myParameter"))
throw new ArgumentException("myParameter);
if(string.Length > 200)
throw new ArgumentException("myParameter is too long.");
with this:
myParameter.Assert("myParameter").IsNotNullOrEmpty().HasMaxLength(500);
I also wrote a stored procedure wrapper that fairly kicks butt. Still using it.
Ages ago, I accidentally wrote a recursive expression parser that let end users enter VB-like expressions to filter data. (Still scratching my head over that.)
And just two years ago, I stumbled into writing an ORM so I didn't have to write the classes to synch the database and the VB.NET classes. I didn't realize that was what I was doing at the time. Silly me.
I wrote a javascript regular expression testing tool. Once done, I found a lot easy to write regex patterns.
Last few months I am building Foglyn, which is an Eclipse plugin which integrates FogBugz into Mylyn. It's helping me to be on top of my cases.
A Validator component for WinForms. It has several builtin rules like check length, if empty or null, if numeric etc..
Also posting here made me realize I wish I've found Mike Hofer's NValidate before starting with mine. Oh well...
I wrote DonsProxy so that I could watch traffic between browsers and the web apps I was building, and between web apps, apps and proxies, etc. I wrote the original at a client site, then did this complete new implementation for the opensource community.
Not really a tool per se, but still: As a web developer, I'm using a lot of frameworks (Code Igniter, jQuery, Blueprint...). And in everyone of those frameworks, I have to put in some methods, styling rules, scripts which I'm using all the time.
Those augmented versions of those frameworks are my tools.
April 2004, I was bored doing consulting work on a very ugly .NET client code base. I took a few hours at night to build a code analyzer that would spot major code quality issues. The tool was named NDepend. It became commercial in Feb 2007 and is now used by thousands of teams worldwide. See the complete Releases Notes.