Madoff’s Programmers


In my last blog entry, I took exception to Mark Cuban’s use of the term Financial Engineering to imply that the word ‘engineering’ was synonymous with the word ‘fraud’. It has been my experience that engineers hate to be wrong about anything, which is no doubt due to their analytical nature, and so as a rule, they are not very good liars. To every rule, however, there are exceptions.

We see evidence on the Internet every day that technically skilled individuals are applying their skills to cheat people out of money. I like to believe that the perpetrators of these scams are a small minority, and it’s only because of the vast reach of the Internet that they seem so ubiquitous. I suppose there may also be cases where otherwise honest programmers are persuaded to help dishonest people because they are desperate for money or want to keep their jobs.

I recently read the book, Madoff with the Money about Bernie Madoff’s Ponzi scheme that swindled investors out of more than $50 billion. The details of the how the scheme was pulled off are not really covered in the book, but all the while I was reading it, I knew that Madoff must have had technical help. Madoff barely possessed the computer skills to send an email. His computer, which basically was set up to display stock ticker data, confounded him. By all accounts, he was of average intelligence, not the financial genius that many people thought he was.

Madoff’s results at significantly outperforming the market year-over-year for more than a decade were baffling to most Wall Street insiders. There are a lot of really motivated people in the financial arena who have access to the brightest minds and best information available and they couldn’t figure out how he was doing it. For that reason alone, many assumed he was doing something illegal. There was even one whistle blower, Harry Markopolos, who contacted the SEC several times including going so far as sending detailed complaint letter describing in excruciating detail 29 ‘red flags’ that indicated that Madoff was either front running stocks (basically a form of insider trading) or, more likely, operating a Ponzi scheme. Markopolos had first contacted the SEC about Madoff in 1999 and several times after that. They ignored him.

About a month ago, two of Madoff’s programmers, Jerry O’hara and George Perez, were arrested and charged for their role in the fraud. They have been subsequently released on $1M bond. Apparently, they are the ones who wrote the programs and operated an old IBM AS/400 computer known as ‘House 17’ that printed out the fictitious statements and reports that made the investment results look real. Below are a few examples of the statements that Madoff investors received.



Apparently, using historical market data, they were able to adjust the market entry and exit dates and ratios of stocks to achieve any return rate that they wanted for the month. Prediction is hard, especially about future events, but predicting the past is easy. But there were a lot more nuances to the fraud than just revisionist history of trading events. They had to make sure that the trades wouldn’t look out of scale with daily volumes or any other anomalies that would set of alarm bells with auditors and regulators. When Madoff had only a few large investors, it was possible to generate statements with this method manually, but as the number of investors grew, so did the complexity of generating the fictitious earnings reports and so it was necessary to employ computer programs to help generate the reports. And of course, any Ponzi scheme must get bigger and bigger to stay alive and to be able to be able to make good on early investor redemptions or it would otherwise collapse. Neither Madoff nor his lieutenant, Frank DiPascali, had any computer skills, so they needed some programmers to help them. I wish I could have been a fly on the wall during those discussions.

I have to wonder what excuse their lawyers will come up with to assert their innocence. Were they just carrying out orders from the higher-ups? Some of the world’s greatest crimes have been perpetrated by people who were just following orders and trying to please their bosses. Their claims of innocence will be challenged because at one point, they decided they didn’t want to participate in the scheme any longer and they withdrew their own funds from the phony investment account, reportedly worth several hundred thousand dollars each. But then they made the mistake of taking $60,000 bonuses and 25% raises to reverse their decisions and continue on with the fraud. That makes it look a lot like taking ‘hush money’. It will be extremely damaging to their claims that they were unwilling participants. Another troubling fact is that they participated in the scheme for more than 15 years. They are now facing up to 30 years in jail.

The irony of all of this is that these programmers had the technical skills to make a decent living, and yet for the sake of making some extra money or to keep their bosses happy, they will lose all of that money defending themselves in addition to facing the prospect of jail time.

Engineers are generally pretty good at understanding the nature of cause and effect relationships, and so it’s hard to explain why a programmer would choose to aid and abet in a criminal activity. Perhaps it was the Svengali-like charm of Madoff himself. After all, he successfully tricked a lot of really smart people into handing over their life’s savings. Maybe it was a case of getting in so deep they didn’t know how to get themselves out it. Whatever defense they use, it should make for some interesting legal wrangling in the courtroom.

Engineering is not Fraud


Mark Cuban, owner of the Dallas Mavericks, recently wrote an article in his blog and used the term Financial Engineering, implying it was responsible for ruining financial markets. I agree with the points he makes, but find his application of the term ‘engineering’ to be a gross misapplication of the word. I suppose it puts engineers in good company, since the term ‘doctoring the results‘ has similar negative connotations, even though doctors are generally thought to be upstanding members of the communities they serve.

I shouldn’t give credit to Cuban for coining the term since there is a short Wikipedia entry for Financial Engineering. It reads:

“Financial engineering is a multidisciplinary field involving financial theory, the methods of financing, using tools of mathematics, computational and the practice of programming to achieve the desired end-results.

The financial engineering methodologies usually apply social theories, engineering methodologies and quantitative methods to finance. It is normally used in the securities, banking, and financial management and consulting industries, or as quantitative analysts in corporate treasury and finance departments of general manufacturing and service firms.”

Not that the Wikipedia is an authority on the topic, but Cuban’s use of the term financial engineering, implying that it involves fraudulently manipulating the market to profit a select group by creating huge losses for true investors, is not consistent with that definition.

Engineers are measured by how well they can discern and explain truths. If you’re an engineer and prone to lying about your calculations, you won’t be an engineer for very long. You could even get someone killed. I’m not saying that it’s impossible for an engineer to lie, it’s just that engineers who do lie usually aren’t very good engineers and generally leave the profession, usually moving on to areas where that skill is better appreciated.

Some professions actually place a high value on stretching the truth and consider it a desirable trait. Marketing comes to mind :-). One of the most prolific writers on the subject of marketing, Seth Godin, actually wrote a book entitled ‘All Marketers are Liars‘, and it became a best seller. I read it and it’s very good, just like his other books. Godin originally wanted to call the book “All Marketers are Storytellers”, but used the word “Liars” to sell more books. That’s an example of lying wrapped in the guise of clever marketing.

Marketing isn’t the only profession where spinning a good yarn can be appreciated. If you’re guilty of some crime, you probably would want a lawyer who could spin the facts to make it look like you’re actually innocent. Used car salesmen are often times thought to be quintessential liars, not all of them, mind you, just the really good ones. If you’re an actor, you have to lie to convince the audience you’re some character that you’re obviously not. So lying may be a necessary skill in some professions, but engineering is not one of them.

That’s why when Cuban uses the term ‘financial engineering’ as the cause the stock market bubbles and current credit crisis, it rings hollow for me. I’m not suggesting he call it ‘financial marketing’, ‘financial lawyering’, or ‘financial acting’, although those terms might more closely align with his implied definition than ‘financial engineering.’

I wish I could help Cuban out by coming up with a new term for the financial fraud that periodically gets perpetrated on Wall Street, but perhaps we should just call it what it really is, that is, fraud, since there’s no reason to tarnish a profession by implying that its name is synonymous with the word ‘fraud’.

Should All Businesses Have a Blog?


I’ve been blogging since 2001, primarily on my own personal website just as a hobby and creative outlet. When I started an engineering consulting business, I didn’t initially include a professional blog on this website. I had just a few static pages. After I converted my static website to WordPress and had a chance to see the effect that just having a few articles can make, I truly believe that most businesses can benefit by having blogs associated with their websites.

Blogging can be used in place of a company newsletter which, in many cases, is just a thinly-veiled form of advertising. In exchange for informing or educating the reader, a newsletter gets something that is very valuable — the reader’s attention. And by providing his attention, the reader subtly gives the company a place in his subconscious mind. The next time he needs a product or service that the company offers, it is a short trip to the subconscious to retrieve the name of that company. But this process wears off eventually, sort of like bathing, and that’s why it needs to be repeated on a regular basis.

The problem with company sponsored newsletters is that many of them lack an authentic voice. If it isn’t on someone’s list of objectives to produce a newsletter, it often fades away after a few issues. And if a newsletter is created by a reluctant author, the results usually speak for themselves. The newsletter can degenerate into a clipping service for stale or unoriginal articles. Margaret McDonald, a communications expert, recently made the observation in her twitter feed that if you send an e-mail newsletter with “Newsletter” in the subject header, you’re saying “This is entirely unnecessary; skip it.”

We are overwhelmed with information today. In addition to conventional sources like newspapers, TV, and radio, the Internet delivers copious amounts of information by email and social networking services like Twitter and Facebook are also competing for our attention. There’s so much information arriving that we sometimes just want to turn it off. However, when we actually want to find some specific information, we want it immediately. This is usually done with a keyword search in one of the popular search engines like Google.

Search engines seem to love blog articles. They give high rankings to fresh content that can’t be found anywhere else. That’s one of the reasons why this website, which is really just a ‘shingle’ on the Internet for my engineering consulting business, has this blog. I can write about topics that are at least tangentially related to engineering consulting and product design and it improves my website’s chances of appearing in search results. If I fail to blog for a month or so, my search engine traffic declines. Hopefully, some of the articles will find an audience and someone in that audience may be looking for an engineering consultant.

To get found on search engines, your website has to have some content for them to index. And if you allow the website’s content to get stale, it will negatively affect your position in the search results.

Unlike high traffic blogs that are written for the sole purpose of delivering eyeballs to advertisers, a consultant may only need a few hundred potential clients to visit a website each month to attract sufficient consulting business. You don’t need to have thousands of visitors per day, but you do need some visitors to attract potential clients. Otherwise, you probably wouldn’t have a website in the first place.

Writing a blog also allows a potential client to ‘get inside your head’ to see if you’re someone who sounds trustworthy and knowledgeable. If you can string a few words into a sentence, and string a few sentences into a coherent paragraph, and then spin a few paragraphs to tell a story, you can stand out from the crowd. It also helps if you can avoid misspellings and grammatical errors, of course, or you may otherwise undermine your credibility and chances of success in attracting clients.

I’m not suggesting that there is no downside to having a business blog. If you come across as being clueless or, worse yet, an egotistical know-it-all, it can work against you. But if you can manage to be genuinely helpful and show mastery of your profession, then adding a blog to your website is a great way to help potential clients to find you. And the best thing about clients finding you through a search is that they may actually have a need for your product or services, or else they would not have been searching for relevant keywords in the first place.

Are Atoms Really the New Bits?


Chris Anderson, editor-in-chief of Wired Magazine and author of the books, The Long Tail and the soon-to-be-released book called Free: The Future of a Radical Price, used the expression, ‘atoms are the new bits’ in his Twitter stream. To clarify what he means, he’s referring to the phenomenon of Open Source Hardware (OSH), a relatively recent trend that follows similar principles to those of Open Source Software. Chris recently ran two open source hardware projects called the Ardupilot and the Blimpduino so he’s quite experienced with open source hardware. Chris also has a great writing style and if Free is as well written as The Long Tail, I expect it will meet with the same kind of success and fanfare as his previous book.

The term open source software can have different meanings depending on its particular open source licensing agreement, of which there are many. In its simplest form, open source software means openly sharing the underlying instructions on how to reproduce a software product. For software to be truly open source, not only should the source code be provided, but the tools needed to modify and re-compile the source should also be readily available. The requirements for providing the source code is often subject to interpretation, and many companies claim to abide by the the open source licensing agreement yet fail to deliver the ability to modify the source. This usually happens because they’ve left out some critical files that may not be bound by open source licensing requirements, and if those files are necessary for compiling, then without them the open source files are not very useful.

Source code for commercial software is usually kept secret to insure that customers will actually pay for the software and to prevent competitors from gaining access to a company’s intellectual property. To insure payment, a company usually needs to prevent others from simply copying the source, compiling it, and distributing it. Most companies selling closed source software will even lock down the compiled bits with digital rights management schemes that make sure the payment is made before supplying a unique passcode to unlock it. Otherwise, the compiled code could be copied and shared freely once the first copy of it was purchased. If a company locks down the compiled code, providing the source code would be like providing the keys to everyone.

There are many different open source licensing agreements and they usually fall into categories somewhere between “do anything you want with it” to “always include my name in the credits.” Some are more restrictive, for example, such as “if you make money with my source code, then you must pay me, otherwise, it’s free.”

Software is virtually free to reproduce and distribute. Hardware is a little different. Before you can do anything with hardware, you not only need the instructions to put it together, but you must also acquire the physical material, i.e., the atoms. And unlike a free software compiler, turning hardware into a usable product requires material and capital equipment that can be expensive. Providing instructions on how to make a hardware product has some value but it is much less of a proportion of the final product’s overall value than it is for software. People are often willing to give away information for free, but if a product includes atoms and if you give them away for free, you’ll go broke eventually. In other words, if I give you information, I can still keep my copy of it, but if I give you something made from atoms, then I no longer have them. This is a critical distinction between atoms and bits.

How much of the value of software is just information or bits? Software is primarily information so the instructions on how to create it would account to nearly 100% of its value. Some would argue this point and say that software always needs user support, basically education on how to use it, and that information also has value. In fact, this portion of the value is where open source software companies generally seek to make their money. For a fee, they will answer questions about the free software or customize it for you.

How about hardware? Where is its value stored? With products made of atoms, there is always some material cost, and the percentage of cost attributable to the atoms can vary considerably depending on the product. For basic commodity products, one can argue that the cost of the atoms is very close to 100% of their value. That’s why farmers have to remain competitive with the current market prices for commodities. Just a few percentage point change in the wrong direction and the farmer loses money. Energy commodities like oil, coal, and gas have similar economic constraints.

Manufactured goods may have a much wider range margins. Luxury goods, for example, have material costs that may be a very small percentage of the selling price. Perfume and jewelery come to mind. But the other costs of selling luxury goods such as demand generation and retailer margins usually make up for the low percentage of the actual material costs.

In the case of consumer electronics, it’s been my experience that the material costs are typically 30-70% of the selling price. There are a few exceptions, of course, such as integrated circuits where actual material costs are a very low percentage of their price. But integrated circuits have enormous capital costs for the manufacturing equipment along with a high development cost for the circuitry. There are even loss leaders, where retailers will sell certain products below cost just to get customers to visit their store. But for the most part, a retailer needs to maintain some margin to make money.

So far, open source hardware has been primarily confined to small circuit boards for hobbyists to build gadgets. The Arduino hardware platform is starting to give open source hardware some new momentum. I think that the Arduino has the potential to be a game-changer because its open source nature is all-encompassing. Not only is the software and hardware open sourced, but so are its development tools. Products based on the Arudino hardware platform are not your typical consumer electronic products. An Arduino-based product might remind you of a science project. Granted, it would be more sophisticated than what passed for a science project a few years ago, but still a far cry from a product that would sit on the shelf at a retailer like Best Buy. In addition to the software that goes inside these gadgets, the schematics, parts list, and artwork files to produce the circuit boards are included, thus helping them to earn a description of truly being ‘open source hardware’.

Providing schematics for electronic products is nothing new. Many electronics equipment manufacturers provided schematics up until the past few decades when they no longer had much value because electronic products became so highly integrated that is wasn’t possible to repair them in the field. Schematics were generally included primarily to help someone repair the product when it failed. The printed circuit files that are now part of open source hardware are a new twist and they have some value to the consumer, but only if one intended to produce multiple copies of a circuit board. Otherwise, it is more economical to purchase the board from someone who is already selling them, preferably with components already soldered to them, due to the economics of building multiple vs. one-off products. But supplying board files does have the effect of limiting the markup one might ask for raw board since with these files, anyone could have a board manufacturer produce the boards, and maybe even add some new features or improvements to it. This additional information, along with the Creative Commons license invites competitors to produce your design and that has been virtually unheard of in the world of manufacturing.

Soldering together a kit of parts was a common activity for electronics enthusiasts before surface mount technology largely replaced the easy-to-hand-solder electronic components. I took great delight in putting together many electronics kits from Heathkit in the 1970’s and 1980’s. Although it is possible with some practice to solder surface mount components to circuit boards by hand, most hobbyists tend to avoid it. It requires patience, steady hands, and some form of optical magnification to do it properly. In production environments, assembling surface mount boards can only be done economically by expensive robotic pick-and-place machines.

I’ve long thought that it would be great to have a manufacturing machine that could take basic raw materials and some downloaded information to assemble them into a finished product. There’s actually a name for this concept. It’s called a Santa Clause machine. I could imagine a machine that took in recycled aluminum, possibly empty beer cans, and produced car parts like the frame, body, wheels, and engine. After downloading the information, the rate at which you could produce a car would only be limited by your rate of beer consumption (or you could collect cans from along side the road to speed things up, of course). In reality, the machines needed to manufacture any product today are highly specialized and would not lend themselves to putting together products atom-by-atom. But this hypothetical machine is fun to contemplate and has been a topic of frequent conversation by those who have trouble differentiating between science fiction and the real world. Many people think that rapid prototyping technology like stereolithograpy has ushered in a Santa Claus machine era, but I’ve been having plastic parts made with stereolithography for more than 15 years and despite solid progress, it’s still nowhere close in terms of cost, speed, and functionality of plastic injected molded parts and doesn’t seem to be destined to ever close that gap. Laser cutters allow you to quickly make some interesting 2D parts, but it has limited applications when making 3D parts. And, of course, you almost always need some metal components and there are no Santa Claus machines for those parts. I am not holding my breath for the Santa Claus machine to arrive.

There’s also the issue of regulatory compliance requirements which is where you have to get the approvals from regulatory agencies like the UL, FCC, CSA, CE, etc.. which basically make sure you don’t:

1. burn down someone’s house
2. electrocute anyone
3. interfere with radio reception

You think that might be easy to pass these certifications, but it’s actually quite involved. That’s what is required for basic consumer electronics products. Some industries have many more additional regulatory requirements. Many open source hardware enthusiasts seem to be blissfully unaware of these testing requirements. Until you’ve actually shepherded a few products through these regulatory approval gauntlets, you haven’t really designed anything the general public could purchase in large quantities. Hobbiest products generally fall below the radar when it comes to regulatory agencies. And when you meet these requirements, it’s important that the end consumer does not make changes to a product that would jeopardize the test results that allowed the approvals to be granted. For those who want to sell products that can be modified by end customers, it’s a bit of a Catch-22.

So are atoms really the new bits? I think open source hardware has definite potential for growth, but like Linux, it will appeal primarily to individuals who thrive on knowing how things actually work, that is, learners and do-it-yourselfers who are technology enthusiasts, or, in other words, less than 10% of the population. The rest of the world just wants to purchase finished products, preferably those that are aesthetically pleasing and carry the brand of a well-recognized company and/or have the endorsement of some celebrity.

When you’re an engineer, designing products for other engineers is technically engaging and fun. You’re basically designing for people who think like you do. Designing for the masses can have an element of drudgery because you may have to intentionally limit products to make them acceptable to non-technical people so they will be mass marketable. Is it possible to do both? That is, can you make something that is technically elegant and capable of modification, yet acceptable as a mass-marketed product? I don’t know for sure, but it’s always fun to dream it is, and if a product like that does come to fruition, it will likely emerge from the open source hardware community. It should be interesting to see what innovations arise from this new trend.

Would you sell foo foo dust?


One definition of “foo foo dust” is a product or feature that really doesn’t do anything, but promises nearly magical properties. It can be used to differentiate one’s products from a competitor’s offerings.

I once worked at a place that made robotic optical libraries. These libraries worked somewhat like a jukebox where the data resided on optical cartridges that were pulled from slots and inserted into drives where their data could be retrieved. Data stored in this manner is called ‘near line’ storage. That is, data that could not be accessed instantly, but could be accessed in under a minute without any human intervention.

The libraries came in sizes from a small desktop model with 16 cartridges and a single drive to products larger than a refrigerator that held 300 cartridges along with 12 drives. One of the innovations of our library was a dual-cartridge picker. This feature saved time because the robot could grab a cartridge and take it to a drive and remove the cartridge that was in the drive and insert the new cartridge without having to store the old cartridge first. That cut the cartridge ‘swap time’ down from about 20 seconds to about 12 seconds, and swap time was very easy to measure and explain to a customer. Thus, it was easy to describe the dual cartridge picker as a competitive advantage. Everyone was pleased.

However, after we’d begun selling this miraculous innovation, someone in the organization reported that the robot was idle about 95% of the time in a typical data room application. Wouldn’t it make more sense to just remove the cartridge from the drive after it was no longer being accessed thus allowing the drive to be empty when the next request to load a cartridge came in? After all, when you got a command to insert a new cartridge into a drive, you had to make the decision about which drive to use so why not empty it during the long periods that the robot was idle? However, if the library did that, it would make the dual-cartridge picker feature useless. This report went over like a ton of bricks. It was not welcomed by the marketing people using the dual-cartridge picker to help sell the product, nor by the engineers whose names were all over the patents. The report was quietly swept under the rug, never to be mentioned again.

We had been unwittingly selling foo foo dust. We had a feature that sounded cool, was easy to explain in a few words, but one that didn’t actually deliver real value. When you dig deep, it’s not hard to find differentiators that sound good when you first hear about them, but that don’t really deliver true value.

It can be very awkward to point out a foo foo dust feature when there is a reverence bordering on worship its perceived value. Sometimes, the head honcho may even have been placed on high because of his involvement in introducing the foo foo dust. So even broaching the subject that you’re selling foo foo dust can be a career limiting move.

What can you do when you suspect your organization is trafficking in foo foo dust? One thing you can do is to bring in an independent party to evaluate the features that are being touted as competitive advantages and, without prejudice, get an honest evaluation on whether what you’re selling really adds value, or if it just give the sales people something to talk about. An independent party can make an evaluation without fear of a reprisal from anyone who has a stake in any potential foo foo dust features. Then at least you’ll know where you stand. If you’ve got features that aren’t actually delivering value yet still costing the customer money, you must ask yourself whether it safe to remove them without losing sales revenue. This can certainly pose a moral dilemma. But in the end, you’re better off avoiding features that add little or no value. The problem with foo foo dust features is that although they can help you sell your products, they will often come back to bite you when your customers find out that they were sold useless features, or perhaps even paid extra because they had been upsold those features.

The long term viability of an organization is best served by avoiding features whose only benefit is that they sound cool. It would be better to ask yourself, ‘If this feature is so great, why don’t any of our competitors offer it?’ And it would also be beneficial to smoke out any foo foo dust in your competitor’s product lines. For example, I’m sure our competitors would have loved to have a copy of the aforementioned report that had been quietly swept under the rug.