Web

Today I also worked on the HTTPD. I have extended the response- and request-classes so that the request can tell the response, that the client (i.e. the browser requesting a page) can handle compressed pages. As the evaluation, if a client understands compression is costly, I added a virtual function to the response so an implementation can inform the request, that it can't compress the contents and so it doesn't have to look into the header. Currently the error response doesn't return any data, so it obviously doesn't need to support compression. The same would be true of binary data like images, but I haven't defined a response to handle that yet, currently I can only send texts.

It has been forever that I really worked on the HTTPD, I have sometimes done some small stuff, but mostly to get it compiling again. Last weekend I managed to update my SuSE 11.1 installation to 11.2 (astoundingly the updater really works) and it finally seems to support my system well enough that I can work. The network is detected with the right speed (in 11.1 the network came up with 10Mb/s) and even my audio card works again.

I am still at the point to add compression to the HTTPD, but I first added something else. As I want to write web services in C++ I need some support to make it easier to create pages. The first addition was, to have a class which creates the HTML-header (containing the DTD, the title and so on). I will extend it later so I can also pass the location of CSS- or JS-files, that way I can write it down correctly once and don't need to remember or look up how to do it. Yes, I know I am lazy.

With that I extended my first test page to contain a bit more text. Next weekend I will try to finally add primitive support for compression.

I haven't worked on the HTTPD forever. Today I finally got it to compile again on a virtual machine. No big changes, just some missing includes needed to be added. I tried to find out, if it would be possible to develop the program in a virtual machine and it seems to work quite well. I exported the directory containing the sources with samba and could then edit the code in Visual Studio (I don't really like to use editors in VMs, they always lag a bit).

So I hope I will now finally make the changes I intended a long time ago. The thing I want to do as next thing is to add compression to the http-connection.

Today I finally uploaded a new entry page to my domain. I worked quite long on the scene and you might understand why I did some of the changes in the last two months (yes, it took that long). There are still some problems with the image, but for now it has to be sufficient. I would much rather have a longer menu to display.

I still don't like to create HTML pages, I think it is a pain. Especially getting it to work in Internet Explorer turned out to be a problem again. I don't even know if it currently works, after I had checked the page in IE 6, fixed the problems (including the whitespace bug) and then checked it in IE7, I found some problems with the way it displayed in Firefox. So I removed some CSS (and hope it didn't destroy rendering in IE) and had to add a conditional comment for two statements for IE. Those I have not yet tested in IE, but I hope it still works.

I just uploaded the changed template. I have removed my small logo, although I really liked it, but it didn't fit in any way to the image. The image is a bit big, I might change the jpg-compression a bit. I am unsure if I should also change the rest, but I think it might make sense to give it a bit different background and I might change the headers of the plugins and the posts. But I am not sure about that, yet. I will see.

What I have found very useful in doing this was firebug, a plugin for firefox. I totally love the ability to change CSS on the currently displayed page. To adjust font sizes, margins, frames and colors there is nothing better. Also to just try, if the text looks better to the right (text-align: right;) it is great as you don't have to find it in the CSS, change it, reload the page. But you just hit the inspect button, click the element and add a new property. Highly recommended.

Only thing it doesn't have is the ability to save the CSS. You can change into the CSS tab and select all and save it. I don't know if that works without flaws, though, I haven't tried it. I don't know what happens if you actually use multiple style sheets. I only made very few changes, mostly had to find a way to align the title of the blog.

I today actually did the first step to prepare this blog for a new design. No I still like this one, I only want to change the header. As I wrote in the todo-list I want to use one of my rendered images at the place of the blue bar. I will also make it a big higher so I can actually fit something into it.

For that to happen I will have to modify the template a bit and also have to find out, if the title text stays readable and how to keep my logo. I know, it isn't much, but I actually like it.

For that I copied the current database and the application to my intranet server and changed some settings so it runs on it. So I can test it before I put it here. It will probably take some days (I hope I at least manage to do that in May).

I recently wrote a bit about the 'Recover Password' stuff you find on many websites and said, that I would target that thing to break into, as I would guess that there I would get much further with a dictionary attack. Yesterday Bruce Schneier posted an essay he had written for Wired about how to choose a secure password. There is a sentence which I found in line with my considerations:

Your encryption program's key-escrow system is almost certainly more vulnerable than your password, as is any "secret question" you've set up in case you forget your password.

Seems as if I am not alone in thinking those things are dangerous.

I didn't intend to write about security placebos again so soon, but I found such a glaring example I have to share. Steffan Esser from the Hardened PHP Project has a blog about security problems with PHP called PHP Security Blog. He recently wrote about a new PHP plugin called ext/filter and suggested to not use it.

As you might know, I still haven't added compression to my HTTP-layer. The reasons are not only that I hadn't implemented parsing to find out, if the client can even understand it, but also because it is a very important part in my eyes and I want to do it right. I want my layer to be very fast and I have hopes I can reach that. One of the traditional speed killers in networking is excessive copying of data. So if I can save on copies, it will have a positive effect, as long as it doesn't end in making the rest more complicated.

I worked a lot on the HTTP implementation today. Now the parsing of requests should be finished. The compression still isn't implemented, because I fist had to implement the parsing of the Accept-Encoding header. This header is used by the client to inform a server if and which compression methods it supports.

I Implemented the connection header. Also fixed some issues with timeouts and sometimes the closing of a connection wasn't detected properly. And I found two big bugs in the receive buffer when testing closing when the maximum number of requests was reached.

Next will be support for compression. I already looked a bit into the zlib documentation. I used it before in my raytracer (I wrote a streambuf-implementation to store my data files as compressed XML). There is something I don't know yet, it seems as if it is possible to compress in multiple stages. This might be an interesting feature some day, as it would make it possible to create a web page where big parts are static and could be precompressed and only the variable parts would need to be compressed for each request. I don't know if that is possible, though.

Finally found some time to implement the timeouts in the HTTPD. Now I am only missing correct signaling of the connection handling stuff (basically the Connection: close when the maximum number of requests that will be served on one connection has been reached) and I would like to implement compression. That means adding the code to compress responses and the correct evaluation of the flags to know if we can send compressed data.

Then I will try to set up the benchmark I wanted to do for some time. I still have to compile a lighttpd with php support. I at least haven't seen any way to only install php without also installing Apache. As I am not really interested in Apaches behaviour I don't want to install it.

Last time I wrote about the things I learnt recently about security. That the security industry has failed, that the common security stuff doesn't work and that there is no sign it will soon. I linked to a nice post about the biggest mistakes you can make when implementing security. I absolutely agree with that and I even have an explanation why it fails, nobody wants to spend thought and time to implement it properly. It has been written again and again and again, security is no product but a process. Perhaps it is easier to understand for me as I worked at the university in a group working in reliability research. There the same principles apply, you can't make a system reliable as an afterthought. It won't work. No, never.

Last time I wrote about the many holes of a web site, there were two more incidents, they didn't fully close the hole to invite yourself to another group. They tried to fix it with a checksum in a hidden field. Obviously they still don't check it on the server side (don't ask me why, it is the only thing you can do to make sure it wasn't tempered with) so I guess in the checksum is the user id of the one issuing the invitation as you needed to be a group moderator (which is easy to become, everybody can start a new group and thereby becoming a moderator). But I guess the group to invite to wasn't included in the checksum. The second was a SQL injection hole on the "forgot password" functionality.

But now for something different. Writing about security holes isn't so much of my interest than finding out, how to not allow them to happen. That was one of the things I did in the week I wasn't very active. There were some articles I read and I have to say, they expressed something I somehow knew but couldn't put into words. It seems, the security industry has managed to somehow get off track. Since it became big business it seems that the interest is more in selling something which we know won't work than in fixing problems. For me the point in time when that became clear was, when Microsoft Corp. started to distribute their malware removal tool. Sorry, but I expect the one responsible for the holes to fix them and not sell me (or for now gift me with) some duct tape.

I think I am confused now. I was being bombed with comment spam in the last months. On Saturday I finally did a backup of the database and the server refused to zip it. I wondered why, but at least managed to download it uncompressed. Well, the thing had 20MB. 20 Mega Byte? How? What?

Ok, after looking into the file, I found where the size came from, it was the log of the spam filter. 17MB of logs. I looked briefly into them and saw, that the captchas were not really effective. As I currently compose a second post in the security line which will be about something I call "security placebo" which is about security stuff which is completely useless, but gives you that warm and fuzzy feeling of having done something, I just switched them off for all comments newer than 30 days and will probably disable them altogether.

I keep an eye on the log, but now imagine my surprise, I didn't get any spam in the blog. None at all. The spam filter seems to now be much more effective and that with a very simple trick. I activated the check on the trackbacks. If a trackback doesn't at least link to my blog, it can hardly be relevant. That seems to get rid of the comments that went through the filters. The log contains one single wrong captcha, the other 114 entries are about other stuff. So I would say, the captchas don't work. I wonder how long it will be until the trackbacks will have an acceptable form. Not too long, I fear.