Page 2 of 2
Re: Does anyone have a local copy of zompist.com?
Posted: Tue Aug 15, 2023 3:20 pm
by WarpedWartWars
Torco wrote: ↑Fri Jul 28, 2023 9:00 am
may i suggest beautifulsoup? manually parsing html is possible, i've done it, but it's almost always simpler to just have someone else's code do it for you.
Ideally I'd use a headless browser, preferably one that supported JS, in a way that I can both control it from the Python program (likely not a problem) and also intercept (and probably also cancel) all HTTP requests from the program.
Re: Does anyone have a local copy of zompist.com?
Posted: Wed Aug 16, 2023 8:30 am
by Torco
pretty sure beautyfulsoup works fine with selenium, which in turn can run in headless mode. I don't know other scraping tools in python for this purpose, but they should be indifferent: still, i suppose it would be easier to ask zomp to give you the files in a pendrive and store it however you were going to store it ? (supposing he's okay with you archiving the site: if he's not, possibly don't do it?)