Short version of the situation is that I have an old site I frequent for user written stories. The site is ancient (think early 2000’s), and has terrible tools for sorting and searching the stories. Half of the time, stories disappear from author profiles. Thousands of stories and you can only sort by top, new, and 30-day top.
I’m in the process of programming a scraper tool so I can archive the stories and give myself a library to better find forgotten stories on the site. I’ll be storing tags, dates, authors, etc, as well as the full body of the text.
Concerning the data, there are a few thousand stories- ascii only, and various data points for each story with the body of many stores reaching several pages long.
Currently, I’m using Python to compile the data and would like to know what storage solution is ideal for my situation. I have a little familiarity with SQL, json, and yaml, but not enough to know what might be best. I am also open to any other solutions that work well with Python.
Is this something that can be run locally without a server? I’m aiming for something as simple as opening the notes app on your phone and selecting a story.
Yes, you can run the web server locally and access it in your browser like any other site. You just wouldn’t be able to access it from outside your home network.
Gotcha. I think I’m aiming for something that runs off a single program. I want to be able to start it up whenever or even transfer it to a drive and use it on something like my laptop. Your idea sounds like it may work, but I’ll have to give it a deeper look.