Book review: Kingpin (a guide to cooperating with authorities for fun and profit)

Sunday, August 5th, 2012

I have read many books about computer security. Many of them tried to shed light on the cybercrime, but all of them failed. Most of the time, people seem to be afraid to touch sensitive stuff, because this typically involves revealing identities of cybercrime bosses and their counterparts – FBI operatives. And without mentioning the names a book becomes too generic and boring.

Here is the book you gotta read. It is the best book written in a decade about the subject. It describes the life of the most notorious American hacker named Max Butler. He is the guy responsible for stealing more than 1 billion US dollars from credit cards. If your credit card ever got abused (I know such people who suddenly found 20 pillows charged to their credit card) then blame this guy.

The book is written with an intimate knowledge of hacker’s daily routine. It also sheds light on his early days, the time when he was growing up, his relationships with other people, etc. This is only possible because the author – Kevin Poulsen is a former black hat hacker himself. The book makes it very clear that hackers are neither good nor bad. Of course, if your credit card suffers then they are obviously bad from your point of view. More generally, however, hackers are responsible for many security tools that help protect networks from some other bad guys. It is pretty clear that some bad people will sooner or later attempt to stage a 9-11-scale attack virtually. This is indeed possible because critical infrastructure is controlled by Internet-connected machines (including flight control towers, power plants, etc.). So, these hackers give specific warning signs to governments: protect your networks or you will be own3d pretty soon.

So, from my point of view, it is very difficult to judge the hacker, even those that steal money. They do harm to individual people, but they are rather useulf to society as a whole. I am pretty sure that without their help (and it is obvious that US authorities are using their help), the US infrastructure would fall victim to some foreign government.

The funniest part in this book is the description of how the hackers actually do their job. Typically, it is a combination of hacking, social engineering, and some luck. For the first part, you need to find an exploit in some popular software such as Internet Explorer. Then, you have to convince the users to visit the page that contains the malicious code. Typically, hackers send some funny e-mails containing a link, which make many people click this link. After that the exploit lands on victim’s machine, and it becomes own3d. What can you find on victim’s machine? Well, it depends. If it is a personal machine of some celebrity or FBI officers, you can find his/her e-mails, documents, plans, schedules, etc. Tons of funny stuff. However, if the victim machine belongs to some enterprise such as local pizza shop then you can get access to customer’s credt cards, if you are lucky. The problem is that many shops fail to abide by VISA security standards which demand credit card number be deleted after processing the order. However, it turns out that many restaurants store YEARS of customer’s traces, which makes them an attractive target for hackers.

After stealing credit card numbers, the hacker passes them to a gang of shoppers, who buy lots of random products, only to sell them on eBay later on. This kind of dirty business involves thousands and thousands of people and is responsible for billions of dollars in losses for credit card companies. But hackers are saying that banks are rich and greedy anyway, so they should suffer a bit (for a credit card theft, the client is only responsible for initial $50 in losses, and the bank carries the rest).

The book makes it very clear that FBI and Secret Service are eagerly interested in hackers’ community. However, it is not just “shoot them all” approach. Rather, the authorities try to infiltrate the hackers’ community (sometimes they even run hackers’ forums), identify the key people and recruit them. It is funny that the identities of recruited hackers is an even bigger secret than identities of FBI operatives. Sometimes, FBI agents from one city are not aware of the fact that the hacker has been recruited by agents in another city, so they attempt to arrest the hacker during his trip. The funniest part begins when they start searching hacker’s house, and then the FBI operatives are told not to search too deeply, implying that the house belongs to the cooperating hacker. So, when the FBI agents arrive with a search warrant, the hacker simply hides his hard drive in a trash bin, and the people who perform the search “fail” to find it. What a funny situation! Everybody understands that all of them are working for FBI (or some other 3-letter agency), but they cannot reveal each other’s identities!

So, the book is a praise to the 3-letter agencies who managed to recruit many hackers. The uncooperating ones are put behind the bars. This is an American story with a happy end. Are there any threats left?

Book review: Cyber Warfare

Saturday, February 11th, 2012

This is a book that I would have never bought myself, but I got it for free through OReilly blogger review program. The title sounds interesting, and the table of contents looks attractive. The main purpose of this book is to investigate whether it is possible to launch a devastating attack without possessing any weapons, by only using an Internet-connected PC. The history is full of unexpected and surprising twists, and it happened many times in the past that country leaders mis-evaluated the possible danger of a certain technology. Quite often, military commanders went on horses to fight with tanks, in which case a large army can fall victim to a small regiment.

The first few chapters of the book are very though-provocative. They try to answer the question posed in the introduction by analyzing which groups are responsible for most attacks. The author makes it very clear that the main threat comes from non-state hacker groups who are indirectly supported by governments. The book never mentions it, but I think those groups are similar to pirates of the medieval ages. They rip off merchants sometimes, but they can be hired to protect trade routes from competing states. As usual, private organizations are much quicker at building up teams with necessary skills, whereas the official government policy is still in its infancy. Also, there is a clear division in what government and non-government organizations do: the informal hacker groups launch attacks, and government agencies try to defend against external threats. The book makes it very clear that this needs to change. The old saying goes that counter-attack is the best way of defending someoneʼs property. Therefore, in order to successfully protect their infrastructure, the governments need to learn how to launch attacks. In other words, the governments need to start building their fleet immediately, rather than keep hiring the pirates.

Many governments realized that long time ago, so in the second half of the book the author presents a comprehensive overview of cyber-capabilities of top 20 or so most advanced countries. This is a bit boring to read, as the book provides way too many details on the internal structure of ministries and agencies. But maybe it is a good reference material for professional strategic decision makers. In addition, when analyzing Russia, I think the book attributes too much power to a bunch of oligarchs. Trust me, they are not responsible for cyber-attacks launched from Russia. There are indeed some covert hacker groups which the book fails to identify.

To summarize, the book is thought-provocative and clearly lays out the looming threats. But in my opinion it fails to identify the driving forces of the cyber war. It is not any particular government or that wealthy guy. Cyber war is still a black market, and the author fails to shed some light on this dark side. Or maybe he just does not want to mess up with them. Because of this elusiveness, I give 4 stars to this book.

Introducing Volkov Commander for Web 1.0

Wednesday, December 28th, 2011

Good old days…. I still miss those IBM PC XT computers…

I would like to introduce my new pet project – Volkov Commander for Web. I wonder if many people know what is Volkov Commander. Maybe some people still remember Norton Commander from good old DOS days? That’s the program I adored during my childhood. No wonder I decided to re-implement it. Of course, the environment has changed drastically, now most popular program is web browser. So, why not implement Volkov (Norton) commander in a browser window, I thought. In fact, the primary motivation for me was being able to clean up the mess that my web site actually was. As the design of my web site was constantly changing, I uploaded way too many pages, images, scripts, etc. Another reason was to link local file system with the web, for example to upload pics to my web site.

Volkov for Web is implemented in JavaScript with some PHP functions acting as back-end. So if you want to use it you have to put the entire package on your web site. Then the websiteʼs file system will appear in Volkovʼs panels. I took advantage of HTML5 file API which makes it possible for a Javascript program to access local file system. With this API it became possible to copy (upload) files from local filesystem to the web site. Traditionally, youʼd have to use FTP to do that, so Volkov definitely simplifies lives of web developers.

Implementing a file manager involves heavy use of UI components: buttons, dialogs, etc. One of my goals was to preserve the original UI of Volkov (Norton) commander. So I implemented my own small UI library. The UI components in the above video do not look quite like those in good old Volkov, but I will keep improving them. The goal is to make two UIs look identical.

I loved Qt framework so much that I decided to give it a boost – port it to Javasciprt! Hence the name: JA QT (yacht). Is it going to stay afloat?

My UI library tries to be API-compatible with Qt library, a toolkit I learned while working at Nokia. Qt is a popular UI toolkit for Linux (think of KDE as an example), Symbian (which was killed by Microsoft). Even though it was thought of as cross-platform toolkit from the beginning, there are some other UI frameworks that are more popular on certain platforms. So, I thought we should give it a boost – and I decided to port Qt to the Web, in other words rewrite it in JavaScript. Hence the name of this new UI toolkit – JAQT (pronounced as yacht): JAvascript+QT. Boy, an ambitious goal it is!

So far, I have ported some basic widgets (QWidget, QPushButton, QLineEdit, QProgressBar, etc), and layouts (QBoxLayout, QVBoxLayout, QHBoxLayout), as well as dialogs (QDialog, QMessageBox, etc.). So I can implement some basic dialogs such as those shown in this video.

Will these two technologies (orthodox file manager + orthodox UI framework) produce a killer app? I guess it depends on the features of this file manager. In particular, going to cloud seems like a promising direction. It would be interesting to make it possible to access Dropbox in one of the panels, and your local filesystem in another. In a longer run, I think I should put this app on Chrome store so that more people can try it, maybe it can become file manager of choice for the Chrome OS.

If anybody is interested in this – do not hesitate to drop me a line.

Reading WIRED Magazine: wake up and build something, everybody

Sunday, May 15th, 2011

I have read a few recent issues of WIRED magazine. It has changed once again. Currently, it does not talk about technology much, despite the fact that many interesting things happened recently in the IT world. Companies such as Apple and Microsoft are all but forgotten in this magazine. Instead, its main focus these days is DIY (do it yourself) movement. WIRED magazine actively promotes all kinds of DIY activities. Almost in every issue there is an article describing a small group of people who were able to revolutionize the world in one way or another.

I don’t really know why they are so obsessed with DIY projects. In one of the issues they claim that open-source development movement is a thing of the past, that open-source is ruled by dictators. More likely, however, that they are taking a broader view on open-source, not just software, but hardware, gadgets, and just everything can become open-source. The magazine states that only a few years ago there were just a dozen of people who could create reasonable DIY projects, so they were not sure it was possible to fill in just one issue of MAKE magazine with interesting content. These days, there are enough projects to fill in 10 MAKE magazines every month. In other words, the DIY movement really took off.

Do these DIY project have any practical meaning? Not really. Most of them are just for fun, the most common project is a weather station that broadcasts info to your Twitter account. Do you need one? Certainly no. But thousands of people have done this for no obvious reason. DIY is a new craze. Or is it just a forgotten obsession from half of a century ago, just like virtualization? (think IBM and mainframes whose OS had virtualization capabilities, but those projects were soon abandoned in favor of micro-computers).

So the DIY age is round the corner. Everybody should get familiar with the following cool technology. Arduino is an open-source hardware board which you can program. Not only WIRED magazine thinks it is cool. Google has announced during I/O conference that Android platform will support devices built using Arduino. This looks like an attempt to standardize hundreds of devices from various manufacturers and allow them to take advantage of Andriod platform. Now, instead of writing Android apps, which there are millions of, people should try to build a cool gadget.

Another cool idea is creating 3D models. Yes, I like to do that as well thanks to Google Sketchup, which opened the awesome modelling world to me given my mediocre drawing skills. WIRED magazine suggests one make another step forward – build your models from real materials: wood, steal, even gold – and open and online shop! To get a real tangible model you need an expensive device called 3D printer. Luckily, they are getting cheaper. A 3D printer is an indispensable tool for those who need to custom parts for their bicycle or want to build a small airplane. It turns out you can do it in your garage nowadays (don’t forget to order the engine, pretty much the only part you need from outside).

But if you don’t have a 3D printer you can use web site such as Shapeways: upload your model, select a material, and you will get a real model by postal mail. Is not that amazing?

But crowed innovation is not limited to software and hardware. It can be just about anything! And there is a special web site that facilitates getting money to fund your next project. It is called Kickstarter. Describe the idea, set a fundraising goal – and just wait for a couple of weeks. Many, really many projects took off through such a community-driven funding. Everybody can donate 1 dollar. And when thousands of people are doing that the result is amazing.

Another topic that WIRED magazine deals with is various high-profile criminal investigations. I don’t know how they are related to technology. I am very surprised to see a lot of such stories in such a magazine.

Optimizing web pages

Sunday, February 7th, 2010

While working on Timeline Builder I noticed that loading a timeline takes a while. This is because Timeline is implemented as lots and lots of Javascript and even though it is minified and all files are concatenated into one the result is still quite big.

Compressing your scripts is a low-hanging fruit in optimizing the size of your web pages. But popular hosting providers don’t implement this feature by default, quite surprisingly.

One low-hanging fruit is to use compression when delivering the files from your web site. Quite surprisingly, this feature is not available by default if you host your web site on popular hosting service such as GoDaddy. But searching on the Internet allows one find out how to do it. OpensourceTutor is a blog with lots of interesting content.

This post describes how to use Apache for our purposes. Basically, it depends on the version of Apache. For 2.x version you have to implement the following .htaccess:


<ifmodule mod_deflate.c>
SetOutputFilter DEFLATE
</ifmodule>

This approach works with hosting provider Logol where I have one of my test sites. However, GoDaddy uses a different version of Apache and another approach is needed. I have tried replacing mod_deflate with mod_gzip but this did not work out.

The approach suggested at OpensourceTutor is to use the Apache rewrite engine. When an HTTP request arrives Apache will check whether a file with the same name but with a postfix .gz exists and if yes then it will serve the compressed file. Therefore, the web developer needs to upload the compressed version of each file that he wants to optimize. The .htaccess code looks as follows:


RewriteEngine on
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.+) $1.gz [QSA,L]

There is a third approach explained here in detail. It works for pages written in PHP. The basic idea is to prepend <? ob_start("ob_gzhandler"); ?> to your pages and that will turn on compression.

These are only few ideas on how to improve your web pages. There are automatic tools that analyze your web pages and make suggestions, most notably YSlow from Yahoo and Page Speed from Google. After reading their reports I learned that there are lots of ways in which I can improve my web site, including

  • Adding expires headers
  • Using entity tags
  • Minifying CSS and Javascript

These ideas mean that I need to maintain two versions of my web site – one which is publicly visible and optimized with compressed and minified scripts and another with uncompressed content which is convenient for development. Then I will need a script that converts the development version into production version. Even though such development configuration seems natural I have not found any tools that facilitate its implementation on popular hosting sites.

Page Speed can generate nice graphs as the one shown above. Each track represents one HTTP request. After I added compression the length of the tracks has been reduced, therefore the results of your efforts are easily measurable.

These two extensions demonstrate the flexibility of Firefox. I have noticed that people achieve glory these days not through a large software development effort such as an operating system or a word processor but through a little application such as a plugin. Typically, plugin runs on a bigger platform along with thousands of other plugins. I have read in Economist that last fall a pizza-tycoon kind of game running on Facebook has attracted several million gamers in a couple of month – an unimaginable rate for the gaming industry alone. But social network makes it possible.

Anyway, there are plugins that are so successful that people build their plugins on top of them. PageSpeed and YSlow are examples of this type of plugin because they are built on top of another Firefox plugin Firebug. Another example of plugin used to develop other plugins is Greasemonkey.

Adding features to SIMILE Timeline

Sunday, January 24th, 2010

I have become interested in adding features to the original implementation of SIMILE Timeline. I have a feeling that it has great potential but despite the years it is been available it did not become widely accepted. In fact, it appeared on a number of cool websites including those of educational institutions, government, etc. However, these are more like showcases of a cool technology whereas I think timeline is something that everybody could use on a daily basis. Given the amount of adoption of social networks one could try to put a timeline there to display your friends activity. In fact, Timeline is available as a plugin for WordPress and I should say it is a very cool thing with lots of bells and whistles but for example there is no such thing as Timeline Google gadget.

One reason of the lack of adoption is the difficulty of sharing. The original implementation of Timeline provides a JavaScript API. In order to use Timeline on your website you have to write approximately 100 lines of JavaScript code which is an unacceptable barrier in many cases. Even if this code is generated automatically and you offer it as copy-and-paste to a user, the code snippet which is 100 lines long is just too much. What is needed is a short solution in a form <script>bla-bla-bla</script> and this is one of the features that I have added.

But there is another, more difficult problem. If you want to visualize RSS feeds that do not belong to your website then you’d run into Ajax same domain policy issue. In fact, I have been running into this problem earlier but then the solution was to implement a PHP proxy that would read the desired RSS feed from another web site and forward it to you. This approach worked fine as long as Timeline stayed exclusively on my site but now we need to share it. Therefore, if a user puts Javascript code on his/her website and this code tries to access an RSS feed through Ajax then it would also need to use a PHP proxy. But PHP is a server-side technology. It is not possible to upload a PHP file to a social network or your iGoogle home page. Therefore, another solution is needed.

I have searched Internet and found a few interesting approaches to the same domain policy problem. One of them is to use AJAX through Flash because apparently Flash is less restrictive. But I was unable to get this approach to work, obviously due to security issues. It looks like the destination site of your AJAX request needs some sort of modifications whereas in my case the location of RSS feed that I want to fetch is totally arbitrary.

In the script tag you do not have to specify actual .js source. You can use any source, for example a PHP script! Just make sure that it outputs Javascript. In other words, if it looks like a duck, quacks like a duck, then it is a duck!

This problem looks totally intractable. But the power of Javascript always amazed me. This time <script> tag came to rescue. Typically, when you specify the URL of the script it is ending with .js, for example <script src="myscript.js"></script> but the funny thing is that you don’t have to! In the src attribute you can specify anything, for example a PHP script that outputs Javascript like this

<script src="myscript.php"></script>

Just make sure that your PHP script outputs Javascript code. The rest is fairly simple: think of the PHP proxy that reads an arbitrary RSS feed and wraps it into Javascript. The PHP script should output a Javascript variable assignment with the RSS feed at the right-hand side of the assignment expression. Then we can use another extremely useful feature of Javascript which is its built-in XML parser:


var xmlDoc=document.implementation.createDocument("","",null);
xmlDoc.load(myrss);

The details of this approach are descried in this OReilly article.

With this technique taken into use it is possible to share a Timeline with arbitrary RSS feeds! The next step is to build a number of plugins, widgets, and gadgets for all kinds of social web sites! Such a bright prospective for the Timeline!

Timeline Builder – my new project

Thursday, December 31st, 2009

Only hours before the New Year 2010 I decided to describe the project that I have been working on this year. In fact, I wanted to summarize my efforts during the whole decade as I started my career in computer industry in year 2000 but I would rather do it later. Now I will present Timeline builder, the project that I have been working on for almost a year.

I started implementing Timeline builder in the beginning of this year. The idea was to allow to display an RSS feed on a Simile timeline. Surprisingly, original implementation of Timeline does not allow to use RSS directly as the event source, either JSON or XML. In order to implement this idea I have written a converter from RSS to JSON using PHP. Then I was using a database to store the URIs of the feeds that a user wanted to display on a timeline. That is, mixing RSS feeds was a required feature from the beginning. That initial version of Timeline builder completed in June was working.

However, after I read a few books on JavaScript I reazlied that it was possible to improve the implementation because RSS is supported natively in JavaScript. Since Timeline is written in JavaScript it was very easy to use RSS as an event source. Indeed, after loading the feed using Ajax it became available as a JavaScript object. This object even has a special field responseXML and because RSS is a subset of XML it is returned in this field as a result of Ajax request. Thus, it was actually very easy to add support of RSS in Timeline.

Because of that, I have extended my Timeline builder with a number of features including:

  • Vertical timelines
  • Icons of arbitrary size. The traditional timeline supports only 20×20 icons which are too small.
  • Labels of arbitrary width and height

After implementing the support of RSS as a JavaScript object I was able to get rid of database. I realized that it is possible to store the data as either JavaScript or in JSON format. These features allow to implement next-level databases, the web databases. I have read an article in CACM that was saying that traditional relational model is not acceptible in many cases for a number of reasons. Thus, using a web database in JSON format is a step forward. I am thinking now that I should implement a web service based on the timeline which would make my web site Web 2.0 compatible.

Timeline builder is still under construction. But I have already added a timeline showing my blog posts to the front page of my website!

Google Charts Builder

Friday, January 9th, 2009

Internet offers lots of information, above any reasonable amount for a normal human being. Thus, consuming information is not that easy. Researchers found out that there are two types of people – those who prefer web pages with few highlights and those who get excited when they see lots of numbers, possibly visualized.

In other words, lack of detail is not always what people are looking for. Personally, I get excited when I see lots of detailed information. Then I bookmark such a web page with the idea that I will visit it again to read it carefully but that happens rarely. I will revisit a web page if it is really important. This is why I like semantic web – it has lots of inter-linked structured information. On my web site I have lots of information represented as charts. Most of it is hidden in a protected zone, for example financial information.

In year 2005 I read an article in IEEE Spectrum. It described the idea of Gordon Bell who worked on MyLifeBits project. The idea is to record all possible information, everything that happens to you throughout the day using a camera. I am trying to implement this idea without a camera, though.The example above shows the number of visits to swimming pool per month.

I need a tool to visualize vast amounts of information. I want to try out different representations – bars, lines, circles, etc. and I need this charts on the web. There are lots of good drawing programs but they are not web-ready.

Google charts bridge the gap between information visualization and web. All the data and metadata are encoded in the URL. Of course, building such URLs manually is not easy. Thus I needed a tool to build charts quickly with different parameters.

I have searched Internet but what I found were tools that asked lots of questions. Instead, give me a chart that is almost ready, I will plug in my data and get the result. In other words, we need an
example-driven tool.

Here is Google Charts Builder. A sample chart is displayed as soon as the page is opened. I tried to make it as nice as possible. Not any type of the chart is supported yet, I will add them as necessary.

I have used this tool to build lots of charts and it allowed me to experiment with scale, chart type, labels, colors, etc.

There are tools for other Google APIs, for example Google Maps builder.

Web APIs are very powerful

Monday, December 22nd, 2008

While implementing Weather forecast on the homescreen of a Nokia device I faced a problem of inserting entries to Google calendar. I have thought of a number of approaches including using Twitter app called TwitterCal that connects the calendar with a Tweet user. In fact, this is quite an interesting application. Implemented as a Twitter user, it receives your messages in a quick add format and inserts them to your calendar. Unfortunately, the application is defunct. Thus the idea was to minimize the development effort. Now I am thinking of a pipeline of Twitter robots. One robot generates weather forecasts. It is connected to the TwitterCal which delivers the weather forecasts to your Google calendar.

But I could not avoid programming. Thus I started using Google Calendar API. As I am used to programming in PHP I wanted to use PHP API. However, that is quite a huge package and it uses Zend, I needed a simpler implementation. Thus, I read the documentation and decided to use good old curl to interact with Google servers. With a minimalistic API allowing only creating and deleting entries I implemented the first version of my project.

I need to deliver weather forecast for any city. Thus I have a search box in which the user enters the desired location. I am using BBC Weather site to search for the location. Here is what the result looks like when I enter New York:

Weather NY

This is not appropriate for a script. In fact, there is a weather forecast just for New York on BBC Weather but there is no direct way of getting there, only through clarifying questions. One idea is to make the script click on the first link on the above page but I have thought of a more elegant solution – use Google Search API!.

We are using Google search on a daily basis. Have you ever noticed that the results are available only in HTML format but not in RSS? Thus, automating search was always a problem. Not anymore, though. Google Search AJAX API allows you to launch queries from within your webpage using Javascript. The idea is that there is an edit box on a web page in which user enters a query and there is a DIV element on your web page in which the results are inserted. Google search is much more robust, it does not ask clarifying questions. If you search for BBC Weather New York the first result is the desired page with NY weather forecast. I am feeling lucky!

When a user of our project enters city name there is no need to display all possible results. We only need the URL of the first result. Thus the trick is to hide the output window to which the Google Search API inserted the results. After the results have been added to the hidden window and the first link has been extracted, we will show that URL on our web page as if we found it on our own! The whole power behind the Google search is neatly hidden on our modest web site. That’s the power of web APIs.

There is a lot of work going on right now on creating new APIs. Their capabilities are limitless. I am thinking of allocating a time slot every weekend to study web APIs that popular web sites expose. A Web API allows you to leverage great powers of those web giants when designing your own web page. You have total control on the look and feel of the results, the external sites shape the data only.

Today I found out that Nokia has adopted the idea of implementing custom homescreen widgets! Here is a contest which goal is to select the best homescreen widget for an upcoming Nokia N97 device. Should I submit Weather on the homescreen there once again?

Weather forecast project

Friday, December 5th, 2008

I have submitted the project to Forum Nokia contest. While working on the project I learned a few interesting techniques. I was surprised that such a simple idea would inspire additional project ideas. In particular, after completing the project I got interested in exploring the following directions:

  • Web search. In the project it was necessary to get weather forecast for any location that user might specify. BBC Weather was the source of information as it has nice RSS feeds with 3-day forecast for any place. There is a form in which user enters city name and then clicks Submit button. The BBC Weather website would then search its database of locations and give the user the forecast for the place that was requested. However, we needed to use the form automatically. Obviously, when Submit button is clicked, a certain GET/POST request is executed. Thus filling in the form automatically was not a problem at all. However, the search mechanism of the website was trying to deliver its best, it asked clarifying questions which was not acceptable for artificial intelligence of a PHP script. For example, when entering New York the BBC search engine asked whether the user meant New York City, or New York airport, or whatever else. But this was happening only for a big city such as New York. In most cases the search delivered the required information without asking any questions. How would one adapt a PHP script to using a web form that asks clarifying questions?
  • Creating screencasts and uploading them on YouTube. I am using Ubuntu and I wanted to make a screencast with text narrated. I quickly discovered Istanbul – a program for creating screencasts. It can capture the whole screen or a certain window. However, a screen of YouTube player is smaller than a screen of a Web browser which I needed to record. I decided to show the global view of a website and then zoom-in on certain areas of the screen. Thus, I needed a video editing software that would allow me to zoom-in on an area of a video. Open Movie Editor was the software that I was able to install. I tried a bunch of programs but they all had library dependencies which Ubuntu package manager was unable to resolve. Also, I narrated description that I needed to stitch to the video. And finally I needed to make sure that the output of video editing software is acceptable for YouTube.

I will describe how I solved each of these problems within next few posts. In general, I found out that each problem was a lot of fun to tackle and that it developed skills necessary in general. For example, while implementing a script that used a search engine I figured out how to use web APIs. These mechanisms are the future of the web which will consist of web applications. Semantic web that operates on structured formats will also make heavy use of web APIs. Automating search might improve the penetration of deep search which is at the moment has very basic capabilities.

I noticed how important it is nowdays to accompany your project with a video. Internet users are short-attention people. They visit a web site, look at it, and then switch to the next web site within a minute. Very few people are able to grasp the text-only information within such a short time frame. This is why multimedia content such as images and video is added to the web. While reading blogs I found out that I like those blogs that have a shiny image or a video. There is a buzz on the web that personal blogging is dead because of commercial bloggers that deliver information in larger quantities. When was it ever that quantity dominated quality? Any personal blog that delivers attractive content will become popular. The problem is that individual bloggers are writing text-only entries. They should include images as well as video. But they need tools in order to do that. In my video story I am going to describe a long chain of tools that allowed me to generate a YouTube video. I spent one week to generate one-minute video.