I have added br tag to it.
When the scraper runs this way, it find's all the br tag which is not inside for example in a p tag.
But when br tag is inside a p tag it won't find the text.
I the case of what is shown on the pic I can't get any of the text inside br.
Are there any chance that You have an easy workaround for this?
Thank You!
Csemid
Dear Mdibaiee!
I have some issues that I can't fix.
Now my tags are looking like this:
tags = ['p', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'ul', 'li', 'span', 'a', 'img', **'br'**]
I have added br tag to it.
When the scraper runs this way, it find's all the br tag which is not inside for example in a p tag.
But when br tag is inside a p tag it won't find the text.
I the case of what is shown on the pic I can't get any of the text inside br.
Are there any chance that You have an easy workaround for this?
Thank You!
Csemid
el.string does not contain all of the text inside the p, only the first piece.
To get all of the text, we need something like this:
full_text = ''.join(unicode(child) for child in el.children
if isinstance(child, NavigableString) and not isinstance(child, Comment))
You might have to import these:
from bs4 import NavigableString, Comment
Please try it and let me know if it works. If it did work for you, please open a pull-request
Hi @Csemid.
You don't need to add the `br` tag to the list as `br` tags themselves don't contain text.
The change necessary on the code is around this line:
https://github.com/mdibaiee/web-scraper/blob/master/index.py#L62
`el.string` does not contain all of the text inside the `p`, only the first piece.
To get all of the text, we need something like this:
```
full_text = ''.join(unicode(child) for child in el.children
if isinstance(child, NavigableString) and not isinstance(child, Comment))
```
You might have to import these:
```
from bs4 import NavigableString, Comment
```
Please try it and let me know if it works. If it did work for you, please open a pull-request
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Dear Mdibaiee!
I have some issues that I can't fix.
Now my tags are looking like this:
tags = ['p', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'ul', 'li', 'span', 'a', 'img', 'br']
I have added br tag to it.
When the scraper runs this way, it find's all the br tag which is not inside for example in a p tag.
But when br tag is inside a p tag it won't find the text.
I the case of what is shown on the pic I can't get any of the text inside br.
Are there any chance that You have an easy workaround for this?
Thank You!
Csemid
Dear Mdibaiee!
Do you think that you will have time for this issue nowdays?
Thank You!
Csemid
Hi @Csemid.
You don't need to add the
brtag to the list asbrtags themselves don't contain text.The change necessary on the code is around this line:
https://github.com/mdibaiee/web-scraper/blob/master/index.py#L62
el.stringdoes not contain all of the text inside thep, only the first piece.To get all of the text, we need something like this:
You might have to import these:
Please try it and let me know if it works. If it did work for you, please open a pull-request