Development Journal of a Raytracer

More problems with the constraints. When I constrained a point to be parallel to a circle and the object as such was transformed, the radius of the circle kept changing. I think I had another forgotten matrix, but now the point doesn't move at all. I have to check what is happening. I really would like this thing to work.

I had a big problem in the HTTPd when I tried to implement the profile page. I want to be able to change multiple settings with one call and so I need to use a transaction. But I never implemented that in my context objects. So today I worked hard to get it into those objects, it should work now, but I haven't tested it yet. But with that addition I should be able to finally implement the profile page.

The next issue is to handle additional data. I want to be able to tell the abstraction I have written for signup/login to handle additional values. I define the way those values have to be put into the table so the post-data can be evaluated and additionally into which column of the user-table the values will be put. That should make it quite easy to handle the most frequent case.

I had a lot of things which occupied me, so didn't do a lot of programming lately. The one thing I did was to look for the problems with the old scene and this took forever. The problems needed very, very long debugging runs and it took considerable time to find the problems. The first was a stupid cast, I had long ago cast a pointer to an unsigned long and later back. When the program was still 32bit it was no problem, but in the 64bit builds that caused a huge problem.

The second problem was, that I had swapped some transformations when I rebuilt a big part of the raytracer to make it fit for parallelization. And finally I had a little bug with dramatic effects. When a shadow hit was on a transparent face, then the hit information was lost. That resulted in the hit being treated as fully transparent, so I lost all shadows for transparent objects. That problem is also solved and I added an assert so I will find problems like that in the future much easier.

There is one problem remaining, the program crashes in a model when I switch antialiasing to higher levels. It seems to be about the same place in all runs, but it is hard to say, because the machine seems to run into heavy swapping and after quite some time the program silently closes and I never caught the last image. In debug builds it runs for hours until it would reach the spot. I didn't have the patience yet to find that bug.

I was a bit busy the last few days. But today I looked into the problems and it seems to get better, although slowly. I found some problems in the object distribution and the plant object. I have changed some of the details of how the raytracer works quite long ago, but it seems I have not changed all of the objects I have. So now I have some stupid problems with them. But I am sure to sort them out.

Today I tried to load an old scene file and it didn't work. I already fixed some smaller bugs I have introduced e.g. I have extended an object and forgot to add code to handle older files. I still have some strange effect I can't localize, but mainly I don't get the scene to render correctly. It uses the object distribution and that doesn't seem to work at all in the raytracer. I will try to find and fix the bug. I guess I should start to add some way to do automatic tests so I at least have a chance to catch these kinds of problems.

I was a bit lazy in the last few days, but today I finally fixed the problem which appeared when I tried to use the constraints in a real model. It was a forgotten transformation when I create the constraints from points of a polyhedron. It should work now and so I will soon try again to use the stuff.

Today I finally tried to use the constraints and snap objects and it didn't work. Somewhere I have lost a transformation so the constraint wasn't where it was supposed to be. I will have to take a look to find and fix it, but I don't expect it to be a big problem.

Today I added the log out page. It doesn't do much, just delete the session. It is rather primitive, it doesn't ask, if it should log out. So it is open for manipulations (tricking someone to click a link to it), but I guess for many types of sites it is ok. I will probably add a second one which uses a form and a security token to make it harder to abuse. Now only the profile page (to change settings like user name or password) is missing.

I also had a good idea how to allow the layout components to access the database. The problem is, that my server is single threaded (by design) and so I can't just tell the thread to yield. But I think I can add the mechanism to the context objects created by the layout components in a way, that simple users of them only need minimally more effort. Basically I need two functions to format and create the response, the first only requesting the context for the layout, the second creating the contents and the response. When the context indicates, that it is not blocking, the second function has to be called directly, else a callback to that second function is registered in the context and when the context has all data it is executed. That should be easy to implement and understand.

Today I made some progress. The login pages is now also fully working and I fixed the layout component. I still have no idea how to do the database queries from layout components, but I hope I will find a way. Now I have two more things to implement and the password thing is finished, the logout page and the profile page. I hope to finish those two soon, I think it shouldn't be very hard now as I have a lot of the issues fixed already.

I have been looking for a hard to find bug. When I tried to use a circular constraint and a plane as snap object I never got a snap. So I started to look into the function, into the nonlinear solver and never quite found the bug. I found one deep in the line search, a small oversight, sometimes the result of the quadratic fit returns a NaN and I had written a check in the wrong way. I had written if (a < 1e-40) instead of if (1e-40 > a). Mathematically they are identical, but the first condition evaluates to false, if a is a NaN, the second to true, which is what I need.

But that wasn't the problem. So I followed another train of thought. The two parts of the equation had one difference I am still not sure matters. The plane function minimizes the square of the distance between the point and the plane, the circle function instead minimizes the distance to the power of 4. I am not sure if that doesn't bias the results and leads to poor convergence. It was easier to write the function that way, but I have now done the function to also use the square of the distance. When implementing it I found out, that the problem was something completely different, I had forgotten a parenthesis in the implementation. So a simple programming bug. But now it works.

But at least I had a good idea while working on this stuff. I think it makes sense, that I mark the snap point, even if it isn't hit yet. It is easy to implement and should help to use the program.

Today I created the objective function and the gradient for the last of the objects, the circle. I think I had a good idea, I wanted to create a compound function which uses the distance to the plane and the distance to a cylinder so that the minimum lies on a circle. But the cylinder would have a rather complex function and I had the idea, that it is not necessary to use it, I can also use a circle as second element. That function is much easier to calculate.

The functions are now in the program, but I didn't have time to test it yet. I also had another good idea, I think I know how to make sure, that the bounds of line constraints are not ignored. I will constrain the point resulting from the minimization and see, if it moves. If it does, then the solution is not acceptable and I should use the original, constrained point. I think, that should fix the last issues.

I added support for the schemas today and the sign up page finally works. I also fixed some of the problems I had with logging errors and finally found the position where I lost the diagnostic message. I still haven't made the change to treat a missing connection to the database as error. That is the next thing I will do and probably work a bit more on logging code.

Then I need to continue with the layout component which prints the user name (and the logout-button) or the login-button. Now that I can log in (by creating a new account) I need to be able to print the user name. I haven't added that yet, but I am glad, as I originally intended to use the database to turn the user id into a name. That wouldn't have worked at all and it would be a stupid idea in any case as the same query would have to be made a lot of times. But I will have to think about a way to make it possible, that layout components make database queries. The problem is, that processing of the request has to be suspended and I don't yet have a good idea. I would prefer a solution which handles this problem without too many code needed in the pages using the components.

I completely removed the snap offset for the constraints. It is wrong in that case and now the code is working as expected. There is still a slight problem, but that I will solve when I have more grasp on nonlinear optimization. I can define, that a line constraint is bounded and that I can not yet express in the equations. I know, that there are ways to do that, but currently I don't think it would be a good idea to start work on that part. When reading the book about optimization the chapters on how to integrate inequalities and hard boundaries were the ones where I had the feeling I didn't really understand what is happening. As I found out, I didn't even understand the parts I found comprehensible, so I assume, that adding support for it would need considerable time.

There was a very simple reason why the sign up page didn't work, the database connection was never established, so waiting for the completion of requests was pretty hopeless. But I think I will make a change to the database object, I it is pretty rare, that you want to queue up requests when there is no database connection, much more likely you would prefer an error.

I also have to add some logging code to the database layer. It would have helped to see, what had happened.

And finally I need to add two new settings to the password abstraction, the names of the schemas. I need the schemas (one contains the data the server can access freely and then other sensitive data only accessible via stored procedures) and I don't really want to use the schema search path.

When I tried to test the code to constrain a point to a line and snap to a plane I found out, that snapping to a line doesn't seem to work. I have the impression, that I broke something with constraining in general. Somewhere the additional offset (the amount the point was moved due to the constraint) is added back to the point. That offset is necessary to guarantee smooth movements and is especially necessary for snapping. Without it you can't move a point away from a snap. The additional offset accumulates the small movements and once they are big enough so the point is outside the snap range it gets free from the snap.