I am having some trouble to focus this morning. So I am going to move as slowly as possible. I lost a lot of sleep last night in the testing of the bulk upload for the Holdings table. But major progress has been made. I was able to get the bulk upload application from Arjun to work, and it did the full update. Even though there are adjustments to be made, I must not let that over-shadow the significant progress that has been made. A pat on the back is deserved. I did very well.
So now for the 2 outstanding issues:
(1) The application should be run 3 times. Pass #1 will take care of the majority of the records. But some will be missed. So Pass #2 and #3 will take care of those. So this could just be a matter of setting the cron schedules. Quite doable.
(2) The application takes too long to update all of the records. I am going to attempt to cut down this time by running 2 applications, each handling 50% of the records. So the spGenAllHoldings will produce 2 csv files. One with the 1st 50% of the records one with the 2nd 50% of the records. Both output files will be named “Holdings.csv” and will be copied to the specific folders. This will happen before 1:30 am. Then the 2 applications will be run and insert their respective 50% of the records. What I have to do is to perform an small scale test to see if they can run simultaneously. I am wondering if the rate of API calls from their combined operation will cause too many errors to occur. This is what I have to test.