Alexey Smirnov's blog

Years are passing by, and each year I am running a few marathons and half-marathons. For each of them I generate a write-up and a picture gallery. But there was no centralized place where I could post my war stories. Finally, I created one, with the help of Drupal, a wonderful site building system.

Here is it. You can see a grid of titles, each one representing a single running event. If you click on a title, a web page showing detailed description opens up. It even includes a map of where the marathon took place. Beneath the map and summary you can see my report. And after the report – a picture gallery (I am always trying to snap a few pics as I go along the way, even if organizers also take pictures).

The funny thing is that it was really easy to build this web site. I did not have to write any code to create a data schema or the UI to enter the data. All of this is done completely automatically. I only need to specify what kind of data I want to store – text fields, geo-location, star voting widget, etc. Very high-level and rather intuitive. The only part that needed some coding was the styling part, that is, when you actually want to specify that the map should be aligned on the left-hand side, and the summary block – on the rigt-hand side. For that purpose I used CSS injector plugin.

In addition to My Marathons web page I would like to build My Readings page which will be dedicated to the book reviews. With the proliferation of book readers I noticed that I am reading more books now. Yeah, these book readers are really useful gadgets, even though I don’t have a dedicated book reader. I am using a rather general-purpose inexpensive tablet manufactured by some Chinese manufacturer called ErgoTech.

I am really excited about Drupal now! I think it perfectly fits to the golden middle on the scale of skills one has to posses in order to build a web site. On the one end there are template-based web site builders which don’t make it possible to build custom web apps. On the other extreme is the pure HTML and Javascript, which takes a lot of time to learn. So – Drupal is what I really needed!

Learn Drupal!

I am spending some of my free time to learn various web frameworks. Recently I decided to take a look at Drupal (I was using version 6 of the framework). So, I installed it after registering account on 000webhost. The installation was smooth.

Then I logged in as admin and started looking around. What I found was a primitive set of tools, namely the ability to create pages and stories. Ok, this looks like basic blogging system. Fine. How about creating a real web site? Menus, custom data types, widgets? I glanced through Drupal’s admin menus but found nothing. Still, Drupal claims to be a web-site building framework.

So how can I build a web site using Drupal?

Oh well, I thought, this framework is less than obvious. Lets get some book and learn from it. So I went to Safari Online and found Pro Drupal development. This book explained how to extend sub-systems of Drupal, such as how to add elements to a page, how to disable a user, etc. Lots of scattered facts, but this book certainly could not convience me that Drupal can be used to build web sites.

I was disappointed by Drupal. Then I thought maybe I should make my requirements more specific. What it means to build a web site? Well, I thought, it should have various sections, such as picture gallery, a list of my favourite recepies, trip reports, etc. So I expected some kind of data store accessible through API, then some way to generate views and so on.

So lets say we want to build photo gallery using Drupal. It should have albums, each album containing arbitrary number of photos. I searched Internet trying to find how to do this. Any free e-books out there? None. Compared to Django which has a dozen of free books, documentation for Drupal is much less complete. Even on the official web site it is rather incomprehensive.

Then suddenly I stumbled upon a few cool free screencasts on drupaltherapy.com. It mentioned that I need to use modules. Yes, M-O-D-U-L-E-S. A-ha, that’s the keyword. Drupal is a pretty creepy system by itself, but modules make it fly! Then I quickly installed a bunch of modules (I wonder why I had to use ftp, could not they provide built-in installer?). I installed CCK, calendar, filefield, imageAPI, imagecache, imagefield, node_reference_url, view, view_attach, views_bonus, views_slideshow, ccs_injector

These modules really brought my Drupal installation to life. Now I can use CCK to add custom content types, views to present data as grids, lists, and slideshows. I can even customize look and feel by injecting CSS (so I don’t have to hack the theme). What is really important is that each module comes with a GUI. For example, the content creation module presents the data type scheme as a list of fields, for each of which you can select type, modify its name and attributes, etc. Views is an even more impressive module – essentially, it makes it possible to query the database without writing a single line of SQL.

So, Drupal aims at a very elusive goal of programming without writing code. As I mentioned earlier, building web site requires knowledge of a whole bunch of programming languages, so it is far from trivial. Drupal bridges the gap between pure mortals who just want to tell the world about their beautiful household, and the intricate world of web technologies. You don’t need to obtain a PhD if you just want to build a web site. Not anymore – thanks to Drupal. Really amazing framework!

And finally, after watching a dozen of screencasts, I found the one I really liked – how to build a photo gallery which supports some kind of hierarchy and references among objects (pics and albums). Here is a screencast – a really nice one!

THE CREW One of the best teams I’ve ever worked in.

After more than a year working on our top-secret project at ITRI time has time to reveal it. Our company has organized a workshop in which many prominent companies such as Canonical, AMD, Dell, and many others participated. So we have created some cool product for ARM server boards.

The workshop was taking place in Daan district in Taipei, which is quite near Taipei 101 building. As it was strating early Monday I decided to come to Taipei Sunday’s evening and stayed in a hostel. The price was pretty cheap, which keeps me surprising but I should finally get used to the cheap prices in Taiwan. It was only 10 USD/night anyway.

I woke at 6AM on Monday and still had couple of hours before start of our workshop. I wished I could go for a morning jog, but I did not have running shoes with me. Instead, I went to McDonalds (oh yes, I am loving it and got few pancakes. Then I spent some time looking for a Russian restaurant to which we were going to go after workshop, but I could not find it, so in the evening we ended up in an American restaurant called Diner.

Then at 8 AM people started coming in, and I helped with arranging tables, etc. On the picture above you can see YF Yuan, our deputy-director who was responsible for this workshop. He was irresistable host. People really like him.

Then we heard the rumor that Mark Shuttleworth migth be visiting us! What an exciting piece of news! Mark became world-famous in 2002 when he went to space as tourist. Obviously, his appearance was not planned in this event. We simply could not dare to expect such a mega-star in our humble event. But he did come and gave a short speech about Ubuntu ARM project. Everybody was very excitied by this unexpected visit!

And I did shake hands with him

MARK SHUTTLEWORTH. Space tourist #2 and founder of Ubuntu.com

Things went on smoothly. I presented our stuff. In fact, we shared 1-hour time slot with people from Virtualopensystems.com. They are small start-up focusing on Cortex-A15 hypervisor, whereas we still focus on Cortex-A9. The advantage of our approach is that it already works, while Cortex-A15 people are still playing with simulator.

After the workshop was over we decided to go to a restaurant. Restaurants in Taipei can be notoriously expensive. That’s why our secretary Phoebe told us earlier that we could spend only 400 NTD per person (around 10 USD). Boy, for that amount of money you can get only 1 sandwitch. So, we took a couple of senior managers with us, and went to a notoriously expensive American restaurant. Well, the guys were polite, so each of them spent only around 1500 NTD That amount of money can buy foods for half a month in ITRI’s cafeteria, and we spent it in just a couple of hours.

But everybody enjoyed that evening. We were glad that our month-long preparation for workshop was over, and it was kind of success. I feel happy that our work seems relevant and interesting to many companies in Taiwan and abroad. Few people have already reqeusted our source code, even though we still want to keep our project close-sourced. Still, we are likely to make more releases, and maybe sell our technology to some bigger company. We will see.

This is a new book by Fred Brooks, the author of much acclaimed Mythical Man-Month. Overall impression: a book of a retired professional who is doing a retrospective of his entire life: recalling how he built a house, how he wrote a book, how he worked in IBM. Such a wonderful, wonderful life. But what does it have to do with most readers who did not happen to know Fred in person, nor visited his marvelous beach house? Nothing. As another reviewer mentions on Amazon, this is a book of wandering mind.

Still, Fred is obviously a distinguished person and listening to him often pays off. But don’t expect to learn any specific technique from this book. There are lots of practical guides on the Internet. The book is a collection of wisdom, most of which, however, I have heard before.

In the first few chapters the book stresses the point that design is iterative process, that it is not possible to get the design right from the very beginning. True. Then he mentions several cases of how the design of complex system was done in a wrong way. For example, designing a military chopper without consulting with pilots. Therefore, even during early design stage it is important that users be taken into consideration. And this is indeed why open-source systems are so successful – because they are driven by users of the product.

Brooks mentions that earlier the designers were actually the users of the product: think of Wright brothers, Ford who rode on the car that he designed, etc. But as the time goes by this happens to change: do you think that space rocket designers are same people as astronauts? Obviously no. And this is going to happen to the software as well, Brooks claims. And I can only add that this is already happening. As a proof I can mention numerous UI frameworks that were designed by UI designers who have little connection with develop teams, not to mention the end users.

Apart from UI design, it is obvious that as the library of software components gets bigger and bigger, the process of building software becomes more like that in assembly factory: take piece labeled S1 and S2, connect them together, wrap them into S3, etc. In such a scenario the designer might not even know how to program. He can use a visual tool to create software. Is this good or bad? I always thought it was good as this makes it possible for people without programming experience to write programs. MIT Scratch is one such example, a hugely popular visual framework not just for kids.

In further chapters of the book Brooks delves into philosophy – empiricism and rationalism. He mentions that abstract math was generated by French philosophers who were mostly into rationalism, whereas applied science was created by Brits who are into empiricism. Brooks claims that software engineering is totally empirical, that is, requires constant verification, and so is the design of software.

Then Brooks sheds light on what are the characteristics of a good design, in his opinion. The major design principles are: orthogonality, propriety, and generality. But those are just general principles. When you design something, you have to make thousands of micro-decisions and the way you make them is called style. How to achieve good style? Brooks mentions the importance of copying other people’s styles. He says one can achieve remarkably good results by just mimicking someone else. As an example he cites music: Respighi’s Ancient Dances and Kreisler’s Praeludium and Allegro (in the style of Pugnani). Listen to the latter, it is magnificent.

He mentions that even great composers such as J. S. Bach spent considerable amount of time studying other people’s works. So, design is a complicated iterative process. As always, documentation plays a very important role. So how can one document the design trajectories? For that tools are needed. Brooks cites a few tools available online, but I cannot say they are mainstream. On the other hand, he fails to mention Mind Maps, a recent tool which is hugely popular among engineers.

To summarize, this book is a message from a successful engineer of previous generation of computer programmers. Can the youngsters learn something from this book? Definitely yes. On the other hand, the book will not fix the problems in existing products, but it can help prevent even more mistakes.

The past summer was the best I had in many years – lots of trips, new friends, and fun. I only had time to read Dostoevsky and to work on my pet projects in my spare time. I kept on receiving Communications of ACM magazine and when three issues have piled up on my bookshelf I thought it was time to read at least one issue. So I picked an issue with a funny monkey shown on the cover and started reading. It has many interesting articles, mostly because they are related to the research I have been doing in Stony Brook. As a graduate student in Experimental Systems Lab I had a chance to work on a variety of systems projects, from databases, to computer graphics, to compilers.

Programmable GPU cards appeared in early 2000s and much has been anticipated. People have claimed that by off-loading computations to them one would achieve significant improvements in systems performance. It was even predicted that people will have 2-3 GPUs installed inside their desktop computers for the sake of improved user experience. That did not happen. After 10 years I have recent Ubuntu distro and a cool GPU card in my desktop machine but the OS is not taking advantage of it. It is possible to take advantage of additional cores or CPUs, for example the OS can schedule different processes to make sure that all available processing power is used, but GPU is out of consideration. According to many researchers, this happens because there is no convenient programming environment that would make it possible to offload some tasks to GPU in a simple and convenient manner as we do it on CPUs.

Computing without Processors proposes a framework called Accelerator developed by Microsoft Research which can compile one source into binary code for CPU, GPU, or FPGA. All right, yet another framework which promises to change the way we program. While reading this article I kept on recalling GPU programming frameworks that have been proposed during the previous 10 years. Most of them have been forgotten. They are:

• Well, in the beginning there was no framework. People wrote GPU assembly code in their programs, and called some libraries passing the assembly programs as arguments which compiled them and loaded into GPU. I participated in a project involving that technique when I was undergraduate intern in Intel in 2002.
• Then there came Cg toolkit from Nvidia. It used same GPU assembly language but provided a stand-alone compiler which could generate some meaningful debug and error messages. I used this toolkit while working on FIR on GPU project in Stony Brook. It was just a course project (btw., the name of the course was Computer Security, hehe). In the end I wrote a tech report, we did not even bother to publish a paper. Years later I discovered through Google Scholar that that report has been cited a number of times, which means other people found it interesting.
• At around 2005 people started to think that incorporating GPU programming into mainstream languages such as C/C++ would make it easier to offload work to GPUs. One example of this approach is BrookGPU project from Stanford. After a couple of years NVidia CUDA came along.

After so many attempts to integrate GPU into programming languages does anybody still think it is possible? Of course, specialized applications already greatly benefited from GPUs, but the question is whether it is possible to take advantage of GPUs on a day-to-day basis. Search for a new programming paradigm continues.

All right, here is another interesting article from CACM: Reputation Systems for Open Collaboration. This article describes work from Google Research on a reputation system for Wikipedia. Can we trust Wikipedia? I think the rule of thumb is that you can use it for the initial learning of a subject of interest, but it might contain lies, so for a more complete study you should definitely refer to some books/articles. These researchers have proposed a new system called WikiTrust available as Firefox plugin. This plugin will highlight individual sentences of Wikipedia entries based on their trustworthiness. Such a great idea! The trustworthiness depends on who wrote that particular sentence and how many article revisions it has survived through. Now I trust Wikipedia!

Here comes another cool article, long but insightful: An Overview of Business Intelligence Technology. It describes ways in which databases can do a better job on handling data and querying it. Indeed, as the cost of storage has decreased, the amount of data has sky-rocketed. I found out that RAM became so cheap that it is possible to load entire data sets into RAM and process queries avoiding disk access. This changes all the related algorithms and data structures, as when we work in RAM we had to take into account cache hierarchy rather than hard drive block placement. I only wonder whether open-source databases such as PostgreSQL can keep up with commercial offerings such as Oracle. In the world of operating systems, Linux is a strong competitor with Windows. In the world of databases, this is probably not the case. On the other hand, Hadoop framework is open-source and it is indeed the best framework for distributed computing.

Few more articles which I found interesting:

• Skinput: Appropriating Skin as an Interactive Canvas. Suggests to use human’s body as touch-based interface. Looks very cool, but the hardware is clumsy.
• The Pain of Implementing LINQ Providers. Interesting applications of compilers for database language integrated query extensions.
• Cognitive Computing. Simulating brain of a monkey on a super-computer.

I don’t play games since graduating from high school. I just don’t have time for that. But recently I have discovered an activity which might look like a game, but it is actually a way to help scientists who want to design drugs. The game is called EteRNA, developed at CMU and Standord. The goal is to determine a sequence of nucleotides (A,C,G,T) to make sure that it folds into a given shape. The game engine uses some sophisticated algorithms to find out how each particular sequence folds. So, given a target shape, you keep trying various nucleotides here and there and see how the shape changes. Once the shape of your molecule matches the target shape, you are done.

So why do scientists need this kind of help? Scientists say this problem is too computationally intensive for computers to handle. Honestly, I really really doubt that. Given the huge amount of distributed processing power at the disposal of scientists, it is not a problem to find the right sequence for a molecule of size 100-200 nucleotides. But there are some reports that claim that humans still do it a lot better than computer. So I wonder why no efficient algorithm has been developed yet. People say there is some intrinsic knowledge to this process.

Still, everybody can recall totally automated folding projects, such as Folding at home. This project produced some really important scientific results. It is funny how nowadays the focus is shifting from distributed computing to crowd-sourcing. Does it mean that power of crowd is stronger than power of CPUs?

When a new player registers, the game offers a few tutorials and challenges, giving a certain number of points for solving each puzzle. Once you accumulate 10,000 points (you need to solve 10 difficult puzzles to achieve that), you can enter The Lab. In this lab people generate a number of designs to achieve a given a shape, then few best are selected by voting, which are then synthesized in tubes. What works well in the simulated environment does not necessarily work in nature, so this feedback is taken into account and the proposed designs are re-evaluated. Based on this evaluation, another iteration of design begins, and so on. After a few rounds a perfectly folding sequence is found.

Even though its scientific contribution remains to be seen, the game is a lot of fun to play. I wonder how people in a research environment could create this kind of product. Typically, what comes out of universities are research prototypes which are functional, but they don’t have bells and whistles that commercial products have. This game is beating any expectations in terms of its UI and features. To me it looks like professional teams of designers, rather than students, worked on it. Probably, that’s indeed the case. Good job anyway, I have had great fun playing this game.

In my spare time I like to read WIRED magazine. A few recent issues were focused on innovation from the United States point of view. According to the magazine, innovation happens within big companies such as Google and Microsoft, as well as smaller startup companies.

Meet Generation Y: The inside story behind Y Combinator. This article describes a startup incubator which is very difficult to get in. It is like, if you are a boss at some big company and you think, Gee, this company is doing lots of junk and I can do so much better, then you can get into Y Combinator and enjoy guaranteed funding in the order of millions USD to build your startup. And this company generated prominent startups such as Dropbox, scribd, reddit, Posterous, out of which I use none. Btw., I think digg is all but dead. Apparently, they are desperately looking for a new big thing, but they have not hit it yet. Those companies mentioned above are cool and nice but few people use them outside of Silicon Valley. So it is like incubator located in Silicon Valley, generating jobs for Silicon Valley, and produces some cool software that local people enjoy.

On the other hand, Bill Gates takes a broader look at the world. After fighting with malaria in Africa, he decided to tackle world’s energy problems. Together with his former colleague Nathan Myrhvold they have designed a new power plant which they want to build completely by 2020. Also Bill Gates says that solar panels are just nice decoration, but they are of little use beyond that.

Another article is dedicated to Kinect hacking. It turns out that Kinect is one of the best-selling computer gadgets in entire computer history – over 10 million pieces sold so far. I will tell you honestly – I don’t know anybody who owns a Kinect! But it seems very cool indeed as it enables numerous applications including 3D scanning and virtual reality.

Google is also a cool company. Android is the only mobile platform that threatens iPhone. According to WIRED, Android advocates complete openness and lack of censorship whereas iPhone is quite the opposite. The article goes beyond technical differences between Android and iPhone. It describes the way Google was able to make a deal with Verizon and Motorola, the latter two companies were in trouble, so they signed up for the deal that they initially didn’t like in order to survive. It looks like Google is advocating mutual use of radio frequencies, which traditional carriers violently oppose. But somehow Google managed to lure Verizon away from its traditional standpoint. And this is indeed a big indication of change going on in the mobile market.

As new people are coming to our center almost every day, I get lots of opportunity to travel. Together we are travelling to various places, either shops or nature, or even some computer exhibitions. The weather is getting hotter and hotter every day, as expected. Nowadays it is around 32 centigrade all day long. And there was no rain during the whole week, only today we had a thunderstorm.

A couple of weeks ago I took my colleague, a Stony Brook graduate student Dilip to a bike tour around Hsinchu. There were lots of small hills in the first part of the trip, so that certainly posed a challenge to us, software engineers, people who spend so much time in their offices. But then there was a flat ride through downtown Hsinchu. We went to a night market (during day time) and ate some delicious local food. Even though we both enjoyed this trip we still did not go there again, because there are so many other trips that we want to make (and we also want to take some rest at home!)

One day I found out that a computer expo was happening in Taipei in the beginning of June. It is called Computex, and it is the second-largest computer expo in the world, next only to CeBit. Many interns got interested in this expo, so we went there on June 2. Another reason I needed to go to Taipei on that day was because Michael, a new employee of my company was arriving on that day. So in the evening I went to the airport to pick him up.

The exhibition was taking place at two main sites: Nangang exhibition hall at east of Taipei, and International convention center near Taipei 101. We went to Nagang first because it contained some software booths. But overall I can say that Computex is still largely a hardware expo. But still, it is quite interesting, and you can get lots of gifts in exchange for a business card. This business card was indeed an issue as it turned out that it was necessary to present one in order to get an entrance badge. None of us had an ITRI business card, so we were seemingly in trouble. However, one guy got his IIT student card, so our whole group of 5 people decided to use it! The guys who were issuing the badges had spent nearly 10 minutes trying to find out who we are, but when we told them that we all just came from India only to attend this wonderful expo they gave us badges and we did not have to pay anything.

Later in the afternoon we went to Taipei 101 area to attend CEO keynote speeches. A number of CEOs from famous companies such as Intel and ARM gave 30 minute speeches describing their companies and their vision on the future of computing. The level of talks was that for general audience, but still they were lots of fun to listen to. I only spent 1 hour there because then I needed to go to airport. I went to bus station and took a bus which was riding for a whole hour. It arrived to Terminal 1, but then I found out that Michael was arriving to Terminal 2. I had to run to another terminal, because by the time I got to Terminal 1 the plane already landed. Luckily, I was on time. Then we went back to Hsinchu on high speed train. It was very eventful day and so much fun is going on!

This is a book that every software engineer has heard of and many of them have read it. And finally I also read it. What can I say… It is a wise book despite the fact that it is almost as old as I am.

The book makes a number of observations on how large projects are being developed. It is funny that large project management was already an issue in 1950s. Many people think (including myself) that during those years programmers were only writing simple programs such as computing some mathematical function or solving an equation. But much more complex software systems were being developed even in 1950s. Therefore, at the time when the book was written a large amount of experience has already been accumulated.

The first claim that the book makes is that developing a reusable software is much more difficult than writing a prototype implementation. Issues such as interoperability testing and documentation can consume as much time as writing code itself.

In the next chapter the author presents what later became the so-called Brooks’ law: adding programmers to a late project can only further delay it. The reason is that in many large projects most time is spent in communication or mis-communication.

Then the author presents an ideal software engineering team: the Surgical team in which each member knows his exact responsibilities and does his job well. Brooks points out that managers should take responsibility for the architecture and convey it clearly to software developers. This ensures that the project has consistent APIs and is well-structured overall.

Despite that many software engineers know these rules in theory, when it comes to practice everybody can recall numerous violations in their owns experience.

So if large teams spend a lot of time on communication, would not it make sense to always use small teams of 2-3 people? It would, as long as the amount of work is not big. But a small team has a fundamental limitation on the amount of man-hours they can tackle. For example, it takes millions of man-hours to implement an operating system, and a small team cannot do it within foreseeable future.

Btw., another interesting observation Brooks makes is that the productivity of a programmer is always 200-300 statements per day, no matter which language is used. This is a fundamental constraint that project managers have to accept. This means that with the current statement-based languages one cannot expect a 10-fold increase of productivity. On the other hand, as programming languages adopt more and more expressive constructs, the productivity of programmers increases anyway.

The book tries to shed the light on the future of programming and mentions various visual efforts aimed at increasing the productivity as well as easing the learning. In fact, projects such as Scratch aim at providing a visual environment in which beginners can build their programs easily without learning programming. So this is still an area of on-going work.

Compared to the current state-of-the art project management technology, this book appears as ancient classic textbook which many people have read but few have listened to. To me it seems that many people do not trust a guy who wrote something 30 years ago in such a dynamic field as software development. This is why people are more willing to adopt Agile methodology whose characteristics are quite opposite to the concept of Surgical team, Vertical system design, etc. However, it is not clear yet whether Agile is an always-win methodology or there are situations in which Brooks’ approach is better. I can only say that project management is getting more and more complicated, just as the complexity of the software itself.

And this book is useful for any kind of project manager, either traditional one or Agile one. This is a must-read classical book.

As I am working in the area of open-source, I am definitely a fan of Linux. However, I also use Windows 7 on my touch-screen laptop. This is why I am subscribed to Windows Secrets newsletter which provides valuable pieces of advice from time to time. Recently a number of very useful programs have been mentioned there.

Soluto - a program that hooks on Windows boot process and benchmarks it. Then it presents a detailed report to you describing programs that were booting and for how long. Then comes the most interesting part. Using a social database of programs, the thing that I anticipated for so long, Soluto decides whether you need this particular program at boot time and explains what would happen if you remove it. In fact, it is not Soluto who possesses such an intelligence, it is the users of this program who contributed to this wikipedia of programs. On my laptop Soluto was able to shave off a whopping minute of boot time instantly. I am impressed!

Another program is Fences. It organizes the icons on the desktop into groups called fences. The fact that each fence is a scrollable window is the biggest innovation of this program. I have never thought that having icons in a scrollable window is so convenient. Indeed, you don’t want to remove icons from the desktop because they represent useful applications. On the other hand, when there are too many icons the desktop looks messy. After installing Fences the desktop looks fresh again. It indicates its willingness to accept more programs. Clean desktop is for clean mind!

Both programs are free downloads.