Developing Programmers .com

Jean Tabaka has posted 78 things I have learned in 6 years of Agile coaching over at rallydev, its worth a read.

I particularly liked point 34, “Agile improves the quality of life for employees”. This matches my experience at work: since we took on Agile methods more seriously, I’ve had much fewer late nights. It did require an Agile expert to help us get on track with it (there’s so many myths about what you can “take or leave” from Agile), but its been very worth it.

Thanks for the link, Korny!

OK, I have not tested this so I don’t know if it will have the effect I hope, but I think it’s a neat idea.

I think technical managers (and perhaps even each developer) should read the commit logs of their dev team.

Not an in depth line by line check; perhaps scan for key points or automate a summary. But know what new functions you have, what functionality and documentation there is, and be able to point people in your team to the right person or place in the code when they are about to do something similar.

Just a thought.

I’ve been hearing a lot recently about “git vs svn”, as though one had to replace the other.

Recently I started reading up on Git for work, and trying it out on a couple of personal projects. It uses a very different development model; I think its not so much a question of “which is better” but “which model better fits what you do?”

svn’s model:

• Fixed group of trusted developers
• Centrally track all changes
• Can control read or write permissions per subdirectory, although write access is required to merge a change back to trunk
• Is against casual rewriting of history; if it happened it happened. You can undo the mistake but a record of it is always there. This is especially handy for providing evidence of IP ownership, and if you release often (eg, using a SaaS model), you can always check on old mistakes to see if they are relevant to a support ticket.
• Each change is pushed to the central server, so users get each others changes more quickly. Some say this is slow; that hasn’t been my experience.
• A merge is applying (typically) all changes to a branch at once, if you want finer detail you’d go back to the history of that branch

git’s model:

• Each user gets a full copy of “history”
• History may be rewritten to just “added this feature”, skipping mistakes.
• Many operations are faster because the information required is all local
• A merge is merging histories, and often followed by complete removal of the old branch
• A “push” to a central authoritative source just pushes your history into a special “pending approval” area, requiring the owner of that central repository to merge changes into the official line.
• ‘Permission’ is managed in a “anyone can take a copy of the whole thing, and make any local changes they want, and what they contribute back must be approved’ model.

As I see it, the development models are different. Svn assumes a small trusted team, and that you don’t want to have to approve other people’s changes, you either give them permission to do it or you don’t. Git assumes a wider contributor base and needs a captain (or “trusted generals”) to approve changes.

You can use both git and svn together (many people do, it gives an ‘authoritative source’ from svn, plus the ability to make local edits that never count if you don’t want them to)

Which tool is better, to me, depends on the nature of your group and how open the project is.

Of course I am new to Git and could be missing some things; feel free to comment.

Please give comments in your own webspace so that I don’t have to moderate them; paste in gitsvn357E84 to your blog post or twitter comment etc, and I’ll find it via: http://www.google.com/search?q=gitsvn357E84

I’ve been adding custom fields to our web API at work and thought I’d share some ideas that came up.

1. Use metadata to automate as much as you reasonably can; eg being able to interrogate an object to find out what calls are available and who may call them is pretty handy.

2. Make it easy for clients to check the api version. At work, we haven’t even officially published our API yet, but already you can query major & minor version number, we have a policy describing how those change thats useful for people writing interface code, and we have a “checkVersion” method where you tell us what version you want, and we report back whether we are backwards compatible to it. Not hard to do, but get that check into the 3rd party clients early and you can save a lot of headaches.

3. Use named parameters, and return values. This makes it easy to add a new parameter while remaining backwards compatible. Depending on your programming language, think “dictionaries? maps? hashes?”.

4. Don’t design around a particular protocol. SOAP, XMLRPC, JSON, whatever. They’re just wrappers, if you know what data you are sending & receiving you can use any wrapper thats convenient for customers.

5. For the sake of customers doing cool stuff, support custom data on key record types. If a customer is doing a mashup and wants to store a twitter ID with a user, for heavens sake let them. My preferred approach is to offer special “get/set custom field” methods, which offer simple name/value pairs that people with API access can set as they wish. Having wished for this feature often in APIs from surprisingly major companies, I made sure we offer it here at work.

6. Publish an explanation of your API calls and sample code to help people work them out. Make sure your examples include raw transactions (eg, the xml) in case people have to get an interface working in a language you haven’t covered.

Just to get the word out, in case it helps me with using APIs in the future, I’ll say again: Custom fields. Tell your friends.

First up, sorry I haven’t posted in a while. Developing Programmers has turned out not to be viable as a business option, so it has become instead a hobby. I will continue posting from time to time because I enjoy it, but less frequently and with less editing (because otherwise its too time-consuming to call “just a hobby”)

Today’s topic, profiling, is related to my past article about “optimising for simple“.

When you first write a program, don’t get too caught up in optimising for speed. If its optimised for simple, then it’ll be easy to re-arrange as you discover how your design works in practise.

Once this is done and you’re basically happy with a design, you might find its a bit slow. The thing to do at this point is run your program with a “profiler”.

A profiler is a tool that tells you where your program is spending its time. This is usually a breakdown by function, block and/or line of code.

This lets you do a sample run that includes the actions you’re concerned about, and find out where to spend your speed optimisation efforts. Often the “hot spots” that slow your program down aren’t where you would expect, so it’s really handy to be able to experiment and find out without bias where it really took its time.

Profilers usually have 2 stages:

Stage 1 is data collection. This involves instrumenting your program to record entry & exit times from subroutines, and usually a lot more information too. With profilers I have used, this has usually been a matter of running your program with special parameters or from inside another program, and getting your program to do whatever it is that’s slow.

Stage 2 is data analysis, which works out total and average time spent in each area of your program and creates a report. The details and presentation of the report varies a lot from one tool to another, but the basic information about where time was spent is pretty consistent.

You should find at least one profiler available for any common language, often more. My current project at work is in Perl so I recently used “NYTProf” to profile it. This particular profiler can generate HTML reports with color coding and explorable “click the function to see the lines in the function” detail, was very easy to use, and found for me in about 2 minutes a mistake I’d made that was seriously slowing down my code. This is the kind of thing that happens with profilers, and the reason you should explore which profilers are available for your language of the day.

At some point you get down to hotspots that are really necessary for your code to get the job done. That’s ok. The main thing is that its not wasting time.

As well as time spent in an area of your program, profilers generally show how often that part of the program was run. This can be invaluable in confirming whether something really matters (eg, perhaps it was a once-off startup cost and you’re not worried about startup time). It can also help with indicating code that would save you a lot of time if you can find a way to move it out of a loop.

A profiler will let you find the “easy wins” for speeding up your code, and can give insight into areas or approaches that are too slow. If you need speed, sometimes you will need to re-design part of your program to work differently, and it might feel like my advice about optimising for simple was for naught. It wasn’t.

First, this design-change situation is uncommon. The far more typical case is once you’ve fixed the “easy wins” your code will be “fast enough”.

Second, how did you know the design change was needed? By having something there to try out. Good thing it’s optimised for simple, so you have a working prototype that’s easy to change and experiment with. Imagine if you’d created complex code to optimise for speed and then found you were wrong about which approach to take?

So, following “optimise for simple” is “get to know your profiler”. Its a lot easier to make your code fast once you have concrete data about where its slow, and a lot easier to fix slowness if the design is simple.

Some starting points:

• For C/C++: Valgrind is both an excellent profiler and memory debugging tool. KCacheGrind can be a nice interface for parts of this functionality.
• For C/C++: GProf is also available although I haven’t tried it personally
• For Perl: NYTProf is made of awesome
• IDEs: Often these will come with (and perhaps even integrate with) profilers. In no particular order: xcode, visual studio, Eclipse.