Ideally I'd use a headless browser, preferably one that supported JS, in a way that I can both control it from the Python program (likely not a problem) and also intercept (and probably also cancel) all HTTP requests from the program.
Does anyone have a local copy of zompist.com?
-
WarpedWartWars
- Posts: 197
- Joined: Sat Aug 28, 2021 2:31 pm
- Location: tɑ tɑ θiθɾ eɾloθ tɑ moew θerts
Re: Does anyone have a local copy of zompist.com?
tɑ tɑ tɑ tɑ θiθɾ eɾloθ tɑ moew θerts olɑrk siθe
of of of of death abyss of moew kingdom sand witch-PLURAL
The witches of the desert of the kingdom of Moew of the Abyss of Death
tɑ toɾose koɾot tsɑx
of apple-PLURAL magic cold
cold magic of apples
of of of of death abyss of moew kingdom sand witch-PLURAL
The witches of the desert of the kingdom of Moew of the Abyss of Death
tɑ toɾose koɾot tsɑx
of apple-PLURAL magic cold
cold magic of apples
Re: Does anyone have a local copy of zompist.com?
pretty sure beautyfulsoup works fine with selenium, which in turn can run in headless mode. I don't know other scraping tools in python for this purpose, but they should be indifferent: still, i suppose it would be easier to ask zomp to give you the files in a pendrive and store it however you were going to store it ? (supposing he's okay with you archiving the site: if he's not, possibly don't do it?)