Python Requests print HTML response variable

Question

I use below script to get the temporary code from server

import requests
from bs4 import BeautifulSoup
payload{
'username':'demo',
'password':'demo'
}
with requests.session() as s:
    r= s.post(192.13.11.100,data=payload)
print(r.text)

No issues in script,

Now, I am getting the output as expected.

<html>
<body>
<script>
var session_key='d2e08508d3';
</script>
<script>
var temporary_data='01';
</script>
</body>
</html>

Now I wanted to get the session_key from the html output.

Please let me know how can I get the variable inside the html

What have you tried? Did you check library like Beautiful Soup? — buran
– buran, Commented Dec 22, 2021 at 8:16
You could try regex with r"session_key=['|\"]{1}([0-9a-zA-Z]{1,})['|\"]{1}" on the html string and retrieve group 1 — benjababe
– benjababe, Commented Dec 22, 2021 at 8:16
@benjababe, although regex MAY work in some very limited number of cases, it's better not to use regex to parse HTML. But you can use regex, ONCE the text from the tag is extracted and then to obtain the specific value. — buran
– buran, Commented Dec 22, 2021 at 8:18

Y. Georgiev · Accepted Answer · 2021-12-22 08:26:33Z

2

You could parse it using RegEx:

import re
regex = re.compile(".*?session_key\=\'(\S+)\'")
session_key = regex.search(r.text).group(1)

Here you can test the regular expression further: RegExr

Here you can find some documentation on the search() method: re docs

answered Dec 22, 2021 at 8:26

Y. Georgiev

6101 gold badge6 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Xu Qiushi · Accepted Answer · 2021-12-22 08:24:36Z

1

Trye this:

import re
from bs4 import BeautifulSoup

test_html = f"""
<html>
<body>
<script>
var session_key='d2e08508d3';
</script>
<script>
var temporary_data='01';
</script>
</body>
</html>
"""
soup = BeautifulSoup(test_html)
session_key = re.findall(r"session_key='(.*?)'", soup.find("script").text)[0]
print(session_key)

answered Dec 22, 2021 at 8:24

Xu Qiushi

1,1711 gold badge7 silver badges11 bronze badges

Comments

év3v · Accepted Answer · 2021-12-22 08:58:16Z

According to this answer : Get JS var value in HTML source using BeautifulSoup in Python

You can do it :

from bs4 import BeautifulSoup
from slimit import ast
from slimit.parser import Parser
from slimit.visitors import nodevisitor

data = """<html>
<body>
<script>
var session_key='d2e08508d3';
</script>
<script>
var temporary_data='01';
</script>
</body>
</html>"""

soup = BeautifulSoup(data, "html.parser")
script = soup.find("script", text=lambda text: text and "var session_key" in text)
parser = Parser()
tree = parser.parse(script.text)
for node in nodevisitor.visit(tree):
    if isinstance(node, ast.VarDecl) and node.identifier.value == 'session_key':
        print(node.initializer.value)

Please reward this answer for the work that he has done: https://stackoverflow.com/a/41020794/17077329

Collectives™ on Stack Overflow

Python Requests print HTML response variable

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related