Friday Facts #47 - CRC fun

Posted by Tomas on 2014-08-15

Hello FFF readers,

another week, another set of news. Kovarex took some more vacation days for a visit to Slovakia where his relatives live. He will come back on Sunday just in time to switch places with me. I will be off for the whole next week, playing Go, hiking and relaxing in the Czech Beskydy mountains. Another fun fact is that the most active forum user ssilk took some vacation as well and he actually came to Prague. I will go meet with him tomorrow.

0.10.7

We had to kind of abandon our "new major feature release every month or two" plan simply because there is too much work to do to stabilize the current feature release. One of the reasons for this is that thanks to the videos by famous youtubers there are now way more Factorio players than let's say a year ago. More players means more gameplay situations and combinations "tested" and that means more bugs found. Even with quite some effort we have been struggling to keep the amount of reported bugs at our forums around 50. The "remaining" bugs are often system specific crashes, edge cases related to our graphics library (allegro) or problems in some complicated game mechanics (especially logistic / contruction network and rail network).

Anyway we have been patiently grinding through the bug list and the result is 0.10.7 which has been released today. It should be one step closer to the stable release for 0.10.x.

Multiplayer progress

In the meantime, the multiplayer branch saw progress namely in topics mentioned in the past Friday Facts. We have fixed the crashes coming from running the multiplayer updates in the separate thread from the render. This also allowed us to improve the game loop to skip unfinished renders (or flips) and continue with the next update instead. I have also spent quite some time looking into determinism issues because this was related to work on 0.10.7 as well (there were couple of bugs reporting that research desyncs the replay).

Currently the fiddling with determinism looks as follows. The game runs in a special mode when it makes a CRC from the whole map every tick. On top of that it actually saves the map with some useful human readable tags to a new file every tick as well. These two operations make the game crawl to at most units of FPS, by the way. A lot of patience is required :) After a while we save the game and replay it. The replay also takes the CRC check every tick. In case this CRC doesn't correspond to the CRC generated during the game run, the desynchronization exception is thrown. Moreover the replay saves the game in the same save format (with humban readable tags) as the running game did.

So now we have two save files with readable tags from the same map tick that we can compare. One is from the replay and the other one from the running game. We also know that this is the first tick of desynchronization (we make the check every tick). Now comes the binary diffing detective work. The ocasional readable tags help us figure out where exactly the saves differ. Since this is the first desynchronization tick the difference is usually very small - typically just a value of one variable. The last step is to sit down, think and figure out how could such a desynchronization happen. Once it is clear what went wrong, fixing the problem is often the easies part.

We have used this mechanism to fix the determinism bugs so far and it works quite well for reproducible scenarios in our local environment. We haven't yet quite figured out how we will handle multiplayer desync problems reported by users. That will be the tough task :D

Rail endings

While working on bugfixes in the 0.10.x branch we keep developing new stuff in the master as well. For instance a neat little thing we call "rail endings" has just been finished. From the gameplay perspective it is a purely visual addition that renders a special sprite at the end of every rail. The train doesn't have a collision with the rail ending (it rides over it). This mechanism has two purposes:

  • The aesthetic feeling is better. There are no more sharp "cut-like feeling" ends for railroads.
  • There is now visual feedback for wrong rail setups. Especially when straight and curved rails interact, it happened now and then that there was an extra straight rail which didn't have any sense. This was causing confusion to the players and we even had bug reports posted based on these setups (for instance when riding train manually stopped in the middle of the curve for no reason - when in fact it reached the end of the wrongly placed straight rail).

Pavel made the sprites for the rail endings and Kuba did the programming integration. You can see an ingame preview below.

GFX greetings from Mallorca

We got an email from Albert saying he is alive and well. He has survived the sailing race and he is now enjoying his time off to relax and gather energy. He will definitely need it because in September he is coming back to Prague to raise the game visuals bar even higher and work on the new content we have had in mind for a while.

The thread for your comments is ready as usual at our forums.