Anyone know the size limit that a JSON file can be when fetch from an external source.
The reason I'm asking is that our power provider (Hydro Quebec) has started maintaining a JSON file that contains winter credit program start and end times that I use to program things like temperature, turn off the water heater and other stuff based on this. The problem is that they are not purging older events and just adding to the file, I'm starting to think that I might hit a wall and won't be able to process them anymore if the file gets to large.
It's already at 9112 characters/ 9kb and it's only the start of winter.
Haven't tried load-testing with a custom app per se, but in case this helps you to know... I've used Rule Machine to successfully pull in a 3MB JSON file into a local variable. While there's some lag in the process, and not intended for much larger payloads (testing with 9+ MB files locked up or killed the RM rule), I just cannot imagine data blocks << 1MB would pose any significant obstacles to the Groovy engine.
Feel free to share a code snapshot in case you'd like resident devs to chime in on best practices.
Thanks for the input, I think I'll make my a bloated JSON file a bit bigger than the predicted maximum number of entries based on previous years before they made this JSON available and test it just to be on the safe side. If the test goes well, I guess I should be ok.
ok well just did a test with a JSON file 79kb size with 81155 characters/2500 lines and all worked as expected when it was parsed correctly.
Best part is that it even saved it in the state variable of the app without cutting anything out or throwing an error. So that is good news for an app I have had in mind for some time now.
