AtoM Web Installer 504 timeout issue
|Target version:||Release 2.5.4|
|Google Code Legacy ID:||Tested version:|
The sfInstallPlugin process has a long running fastcgi call (loaddata) that sets up the db schema, loads all settings, and populates the ES index on a new AtoM install. On some systems this task runs longer than NGINX's fastcgi_read_timeout default of 60 seconds triggering 504 timeout errors in the browser during the install process. Adding a fastcgi_read_timeout value of 120 in the nginx/sites-enabled/atom file in the index.php section gets rid of the error. On a test VBox VM it was taking about 1 min 15 secs to complete and producing 504 errors in the browser.
The long running server side tasks are run sequentially here:
Break these tasks up so that they can each complete in less than the default fastcgi_read_timeout setting of 60 secs.
Possible to break out into separate controllers? Maybe remove the ES index task so that this can just be run manually afterwards using the CLI task (perhaps it's doing more than just a search:populate?)
It is possible to work around this 504 if it appears by just keying in the next uri in the sfInstall process. See thread here:
#3 Updated by José Raddaoui Marín about 1 month ago
- Status changed from New to Code Review
- Assignee set to Mike Cantelon
- Target version changed from Release 2.6.0 to Release 2.5.4
After some analysis we came up again with the same idea of splitting the load process in different actions. However, the initial tests of that implementation have shown that it's not a viable solution, while the search population part takes around one second, the MySQL data load takes between 2 and 4 minutes.
Nevertheless, after realizing that the same process takes around 30 seconds when it's done through the purge task, I could find that removing the transaction filter from the web installer reduces the data load time in there to the same 30 seconds. I still want (need) to find the reason why removing the transaction filter reduces the data load time, but the code is ready for review in:
#4 Updated by José Raddaoui Marín about 1 month ago
The problem seems to be that we were mixing Data Definition Language statements (CREATE and ALTER tables) with Data Manipulation Language statements (INSERT and UPDATE) in the same transaction. More information:
#5 Updated by José Raddaoui Marín about 1 month ago
I've updated the web installer fix after the latest findings. In the end, splitting the DDL and DML statements in two different request cycles, apart from splitting the data load time, it doesn't require to remove the transaction filter. During the process, I found a problem with the ES configuration management by the web installer, which is addressed in the same commit.