Copy a Linux system over the network, with no intermediate files

27 October, 2008 (21:41) | Linux | 1 comment

I had an old Red Had Linux system I wanted to convert to a Virtual Machine. The typical method recommended by most people is to back up the system, then restore it inside the VM. The problem I had with that is that you end up wasting a lot of time creating an image you don’t need, plus you need a storage device large enough to hold the image which can be accessed from both systems.

My solution was to create the VM with a virtual drive the size I needed. Then use a Knoppix image to initially boot the VM, once you get to a command line, use the commands below.

On the receiving server:

sudo nc -p 2222 -l | dd of=/dev/hda

On the system to be copied:

sudo dd if=/dev/hda | nc 2222

If your CPU on each device is faster than the network, you could also gain some speed by piping the data stream through bzip2. E.g. “sudo nc -p 2222 -l | bzip2 -d | dd of=/dev/hda” and “sudo dd if=/dev/hda | bzip2 -c | nc 2222″. Whether or not it’s worth it depends on the speed of your CPUs and the speed of your network.

It’s not a requirement that you boot the system being imaged from a CD, nor that you shut down any services. But, if you don’t, it’s possible you might end up with some files that aren’t in a valid state from the application’s point of view. There’s no real method I know of to monitor the progress of the copy, but you can run “fdisk /dev/hda” to verify the copy at least got started. If the copy started OK, then you’ll see partition information the same as on the system being imaged. You can also run “ifconfig” and watch the “RX bytes” field to get an idea of how much data has been transferred, but the counter will eventually overflow and start back at 0.

0×8007232b error when attempting to activate Windows Server 2008

9 August, 2008 (14:15) | Windows | 0 comments

I had just ran into an issue attempting to activate my copy of Windows 2008 Server downloaded from MSDN. The error expands to “DNS Entry not found”, although all networking features work fine. The issue was that it was attempting to validate against an Activation Server rather than Microsoft’s website. The solution was to open “System” in the Control Panel, down at the bottom of that window is a link titled “Change Product Key”. I entered my key from the MSDN website and Windows activated automatically after the key was entered.

The difference between window.setTimeout() and window.setInterval()

1 August, 2008 (12:49) | JavaScript, Programming | 1 comment

From what I’ve seen, the problem most people have with the difference between these two JavaScript functions isn’t knowing what they do, but it’s knowing that they both exist. I’ve seen lots of people constantly call setTimeout() calling the same function over and over where the last line of the function calls setTimeout(), but setInterval() is probably what they really wanted, they just didn’t know it existed. But I’ve also seen lots of code that calls setInterval() to execute a function after a specific delay, and the first thing that function does is call clearInterval(), so they obviously really wanted the functionality of setTimeout(). I guess it’s mostly a result of people finding one thing that gets the job done, and never looking for anything else.

The difference between the two functions is fairly trivial. setTimeout() executes the function only once, unless you call it again; and setInterval() continuously calls the function until clearInterval() is called. The only disadvantage to using setTimeout() when you really wanted the functionality of setInterval is that the time between each call will vary depending on how long it takes your code to execute (and sometimes you actually want that feature). There is some danger in using setInterval() and hoping your code is fast enough to call clearInterval() before the interval time passes. In such a case, the next call is queued and will execute your function any number of times depending on how many times the interval time passed while your code was running.

Programming vs. Engineering vs. Mathematics.

20 June, 2008 (17:28) | Programming, Humor, Computer Science | 0 comments

An experiment was conducted in order to study the differences between programmers, engineers, and mathematicians. One programmer, one engineer, and one mathematician were all locked in separate classrooms for one week. Each classroom contained all of the normal things you’d find in a classroom: chalkboard, chalk, desks, etc. In addition to that each classroom contained water, one can of food (enough to feed a person for a week), but no can opener. At the end of the week, they opened the programmer’s door first. The programmer walked out. When they looked in the classroom they saw the can beat to hell, but opened. When they opened the engineer’s door, the engineer walked out. In the classroom were a bunch of equations and diagrams written on the board, one small dent in the can which was all that was required to open it. Lastly they opened the mathematician’s door. No one came out. They looked in and saw the mathematician laying dead on the floor, the can of food untouched, and a long series of equations on the chalkboard. But at the end of the equations was written the phrase: “A solution exists”.

Getting your extension to work with Firefox 3.0

18 June, 2008 (13:20) | JavaScript, Programming | 0 comments

When you develop a Firefox extension you’re required to specify the minimum and maximum versions of Firefox which your extension is known to work with. This is done by specifying a minVersion and maxVersion in the install.rdf file inside your plugin’s archive. Since my ClearSkyPlugin isn’t likely to break in any future version of Firefox, I attempted to bypass the version checking by setting the minVersion to 1.0, and the maxVersion to 9.0.
Of course, today Firefox 3.0 was released and my extension was detected as incompatible. After some twiddling I managed to figure out I had to set the minVersion to 1.0.* to get things to work.

If you’re having trouble getting your downloaded extensions to work, you might just disable extension version checking altogether.

Simple speach example in C#.

12 May, 2008 (15:57) | .Net, Snippets | 0 comments

This uses reflection to avoid the clerical task of adding a project reference:

System.Type t = System.Type.GetTypeFromProgID("SAPI.SpVoice");
object o = System.Activator.CreateInstance(t);
t.InvokeMember("Speak", System.Reflection.BindingFlags.InvokeMethod, null, o, new object[] { "test", 0 });

If you don’t mind adding a reference, then the whole thing can be done in one line:

new SpeechLib.SpVoice().Speak("This is a test", SpeechLib.SpeechVoiceSpeakFlags.SVSFDefault);

Beware of System.nanoTime() in Java

25 April, 2008 (16:45) | Java, Programming | 3 comments

The documentation for System.nanoTime() (and the function’s name itself) leads developers to believe it’s a much more accurate timer than anything else Java provides. Depending on which set of documentation you read, it claims to use “the most precise available system timer”. While I can’t speak for any architecture other than x86, I can state that System.nanoTime() is broken for multi-CPU or multi-core x86 systems, and generally unreliable on all x86 systems. The problem lies in the RDTSC instruction which retrieves the number of CPU ticks since the CPU started. On multi-core systems, each core will have its own tick count, and they will not match, so every time your process switches CPUs, you get a different measurement. The issue is compounded by the fact that some power management systems actually alter the CPU’s frequency to save power, which breaks the functionality even on single core, single CPU systems.

In general, do not use System.nanoTime() in any program which you don’t plan to strongly control how it is run, and the environment it is run in.

Implementation of DES in C

16 March, 2008 (23:31) | Cryptography, Security, Code, Computer Science | 0 comments

The difference between Java’s “final” and C#’s “const”

15 February, 2008 (20:01) | .Net, Java, C#, Computer Science | 0 comments

Final in Java is used to declare a class that can’t be subclassed, a method that can’t be overridden, or …

A final variable means the value won’t change, but the value can still be determined at run time:

final int i;
if ( j > 0 ){
   i =1;
} else {
   i = 2;

You get a compile time error if the final variable isn’t definitely unassigned anytime you assign to it:

final int i;

In C#, a “const” must be determined at compile time.

Learning to program: A complete curiculum

17 November, 2007 (16:28) | Programming, Computer Science | 0 comments

I’ve seen many, many posts in forums asking about some good books to read to either get started in computer science, or what next steps to take after having read one intro book on a specific language. Of course none of them state what their goals are, other than they “want to learn to program”. But someone who just wants to write an occasional one off application for fun would take a vastly different route than someone wanting to earn a living as a programmer. This post is for the latter group, those who need in depth knowledge of how to design and develop systems, and are willing to accept that a lot of the process of getting there require a lot of hard work.

I’m not going to recommend specific books for several reasons: 1) Which books are good changes with time, the books I used to cut my teeth on are mostly out of print. 2) There are likely several books which are more than sufficient in each category, so there is no best book, each individual should choose books which suit their needs. 3) There are a lot of resources on line which can augment a book, for example some of the on-line course lectures, and it would be best to stick with the book(s) used in the course. 4) I have not read every book in every category, nor do I even know enough people for our combined readings encompass every book in every category, so it would be impossible to fairly represent every book. 5) I don’t recommend anyone go out and buy all of these books at once and try to read them in one sitting, rather they’ll have to be studied over time, so a better book might have come out while you were working your way through a different one. I would recommend individuals read reviews of whatever books are popular at the time they’re in the market for them.


The following topics should be considered required reading for everyone. No matter which type of programming you plan to get into, the following topics will provide a foundation to build on. The order of the books is not all that important, though you would obviously want to learn the basics a programming language as a first step. I have tried to put them in order so that it wouldn’t hurt to follow through them in the order listed.

  • Basic language book: This can be something as basic as “Teach yourself x in y hours/days/weeks”. Just don’t expect to learn it as fast as the title says. The language you choose shouldn’t be of too much concern, by the time you’re ‘done’ with this program, you’ll likely have become familiar with many different languages. A very common mistake is for beginners to put too much weight into which is the best beginner language, just pick one and go.
  • A more in depth language book: The basic book probably skipped over a lot of the features of the language. Choose a book which goes over the language in depth. There are many introductory books which do a good job of presenting the important parts of a language, so you might be able to get by with a book which combines #1, and #2.
  • Data Structures and Algorithms: This is where you start to learn how to actually get work done, and get it done efficiently. This book, and the books that follow won’t focus so much on the programming language, but higher level concepts of how to design programs.
  • Programming Languages: There are lots of languages out there, each with their strengths and weaknesses. There are also many common features among different languages. This book will help you decide which language features are available and which one is the best tool for whatever task you’re about to undertake.
  • Computer organization: This book will definitely include assembly language as one of it’s topics, in fact many books titled “Assembly Language” will suit the bill nicely. The purpose of this topic is to get an idea of what’s going on under the hood to help you design more efficient systems, as well as explain why somethings are or aren’t possible.
  • Operating Systems: Odds are you won’t be working directly with a machine, but will rely on an operating system to make working with the different parts of the machine easier. This book will likely assume you’re somewhat familiar with assembly language. Note, this is not a book titled something like “Teach yourself Windows XP in 21 Days”, but is a book which doesn’t focus on a specific operating system. It will concentrate more on the underlying concepts of how operating systems are designed. That’s not to say a book on the operating system you’re using isn’t a useful resource though.
  • Discrete Mathematics: This book will almost definitely not contain any code, in fact, it might not even mention computers. It will abstract out the abilities of the computer into mathematical terms. This will allow you to transcend the specifics of the machine you’re working with to design efficient solutions to problems, while still keeping the solution as something that can actually be implemented on a computer.
  • Networking: This shouldn’t be a “Network Programming in <Language X>”, but a book which focuses on the underlying principles of network communication. The book will likely not contain a single line of code, but it’s OK if it does. I wouldn’t recommend a totally abstract book, rather something that focus on TCP/IP with explanations of why certain things exist (or don’t exist) in the protocols would be OK.


For persons interested in game programming vs. those interested in business applications, the path taken will be different. Since my background is mostly in business applications, I can only recommend topics along those lines. Not all of these topics are specific to business programming per se’, but I don’t necessarily consider them to fall under the “required for everyone” heading. These topics are in no particular order and can be read concurrently with each other. And I would even go so far as to say you can read them concurrently with books in the “required” list.

  • SQL and database design: Almost every business application will involve the use of a database of some sort. It is important to be fluent in the use of databases in order to work efficiently.
  • Specific databases: Oracle and SQL Server, MySQL, etc: There are several different popular implementations of databases available, each with it’s own proprietary extensions to SQL. Unless you’re starting your own business, odds are you won’t get to choose which database implementation you get to work with, so it’s a good idea to be familiar with all of the most popular ones. Many books on a specific implementation also include a good into to SQL and database design, so you can combine this with book #1 on this list.
  • Design Patterns: Many programs share similar ideas, so much so that it’s a good idea to give certain ideas or “patterns” names. This allows you to discuss and explain patterns of programs with other developers at a much higher level. Additionally design patterns serve as examples of good design, and will save you the trouble of inventing them yourself. There is also a concept of “anti-patterns” (also called pitfalls) which are examples of common bad design, which can also be of great help. I would also like to warn you specifically of one anti-pattern, and that’s the overuse of design patterns. I’ve seen many programmers get too excited over design patterns that they want to use them everywhere, so it’s important to know when and where to use them.
  • Human Computer Interaction: Not all business applications involve a user interface, but you’ll likely end up writing many applications that do. Being able to develop a good, intuitive user interface is a very difficult task in most situations, but is one of the most important features required in order to make sure your application is actually useful.
  • Object Oriented Design: This is mostly a required concept, and is generally covered somewhat in even basic books on programming languages which are base on object oriented concepts. But it’s a good idea to study specifics of object oriented design in depth.
  • Computer Security: This is a definite must for any business application. It’s easy to be too naive and think no one will attempt to crack your application, or that they couldn’t possibly figure out your home grown obfuscation technique. Studying security will help you be skeptical about the security of every piece of your application and give you common tools to use to develop strong, robust, reliable systems.

Learn other languages:

Knowing only one programming language is almost like owning only one tool in your garage. I’m not saying you need to be a guru in a dozen different languages, but you should be fluent in at least a few languages, and a guru in one or two with the ability to become a guru in other languages quickly. While every business dreams of having all of their applications written in just one language, it’s something that pretty much never materializes, and certainly not for any successful business since the life of the business will certainly exceed the life of the language their applications are written in. So it’s an important marketable skill to be fluent in many different languages, as well as having the ability to pick up new ones without taking a week long training course. Just as an example, on any given day, I might program in as many as 13 different languages: VB6, VB.Net, C#, Java, Perl, C, C++ PHP, JavaScript, VBScript, Python, PL/SQL, Transact-SQL are all languages I use daily (but I don’t ask for your pity). I recommend knowing languages which encompass all of the following features (most languages will have several of these properties, but the idea is to learn several languages):

  • Object oriented language
  • Scripting language
  • Language with memory management built in.
  • Language without automatic memory management (C would be my recommendation)
  • Machine language
  • Static typing
  • Weak typing

Continued Learning

Once you’ve built a foundation, you can’t stop learning there. Computer programming is a field which is constantly changing and will require continued effort towards learning new and different techniques. So, if you don’t like learning new things, this probably isn’t the field for you. There’s a good chance that most of the techniques you learn today won’t be of much use 10 years from now, and whatever you’ll be doing 20 years from now will be barely recognizable as programming to a programmer today. Many of the underlying concepts change much more slowly than the actual technology, which is why it’s important to learn them. When you understand the concepts, it’s much easier to find a tool to make it happen.