Parse html failed with BeautifulSoup and requests, I can't figure it out, can anyone tell me why?
Here is my code:
import requests from bs4 import BeautifulSoup r = requests.get(self.url) soup = BeautifulSoup(r.content) print r.content # Here, I got output of the html print soup.text # Here, I can't get any output, which puzzled me.
It may be what reason/reasons caused this?
Thank you guys!
I got it!
Because the BS didn't parse the document correctly, so it lost some content.
The solution is:
soup = BeautifulSoup(r.content, 'html.parser')