Commit c1a5cc0a authored by Ian Briggs's avatar Ian Briggs
Browse files

ch10

parent 87e35e09
Tests for WBE Chapter 10
========================
Chapter 10 (Keeping Data Private) introduces cookies.
>>> import test
>>> _ = test.socket.patch().start()
>>> _ = test.ssl.patch().start()
>>> test.NO_CACHE = True
>>> import browser
Testing basic cookies
=====================
When a server sends a `Set-Cookie` header, the browser should save it
in the cookie jar:
>>> this_browser = browser.Browser()
>>> url = 'http://test.test.chapter10/login'
>>> test.socket.respond(url, b"HTTP/1.0 200 OK\r\nSet-Cookie: foo=bar\r\n\r\nempty")
>>> this_browser.load(url)
>>> browser.COOKIE_JAR["test.test.chapter10"]
('foo=bar', {})
Moreover, the browser should now send a `Cookie` header with future
requests:
>>> url2 = 'http://test.test.chapter10/'
>>> test.socket.respond(url2, b"HTTP/1.0 200 OK\r\n\r\n\r\nempty")
>>> this_browser.load(url2)
>>> b'cookie: foo=bar' in test.socket.last_request(url2).lower()
True
Unrelated sites should not be sent the cookie:
>>> url3 = 'http://other.site.chapter10/'
>>> test.socket.respond(url3, b"HTTP/1.0 200 OK\r\n\r\n\r\nempty")
>>> this_browser.load(url3)
>>> b'cookie' in test.socket.last_request(url3).lower()
False
Note that these three requests were across three different tabs. All
tabs should use the same cookie jar.
Cookie values can be updated:
>>> browser.COOKIE_JAR["test.test.chapter10"]
('foo=bar', {})
>>> test.socket.respond(url, b"HTTP/1.0 200 OK\r\nSet-Cookie: foo=baz\r\n\r\nempty")
>>> this_browser.load(url)
>>> browser.COOKIE_JAR["test.test.chapter10"]
('foo=baz', {})
Testing XMLHttpRequest
======================
First, let's test the basic `XMLHttpRequest` functionality. We'll be
making a lot of `XMLHttpRequest` calls so let's add a little helper
for that:
>>> def xhrjs(url):
... return """x = new XMLHttpRequest();
... x.open("GET", """ + repr(url) + """, false);
... x.send();
... console.log(x.responseText);"""
Now let's test a simple same-site request:
>>> url = "http://about.blank.chapter10/"
>>> test.socket.respond(url, b"HTTP/1.0 200 OK\r\n\r\nempty")
>>> url2 = "http://about.blank.chapter10/hello"
>>> test.socket.respond(url2, b"HTTP/1.0 200 OK\r\n\r\nHello!")
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
>>> tab = this_browser.tabs[0]
>>> tab.js.run(xhrjs(url2))
Hello!
Relative URLs also work:
>>> tab.js.run(xhrjs("/hello"))
Hello!
Non-synchronous XHRs should fail:
>>> tab.js.run("XMLHttpRequest().open('GET', '/', true)") #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
_dukpy.JSRuntimeError: <complicated error message>
If cookies are present, they should be sent:
>>> browser.COOKIE_JAR["about.blank.chapter10"] = ('foo=bar', {})
>>> tab.js.run(xhrjs(url2))
Hello!
>>> b'cookie: foo=bar' in test.socket.last_request(url2).lower()
True
Note that the cookie value is sent.
Now let's see that cross-domain requests fail:
>>> url3 = "http://other.site.chapter10/"
>>> test.socket.respond(url3, b"HTTP/1.0 200 OK\r\n\r\nPrivate")
>>> tab.js.run(xhrjs(url3)) #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
_dukpy.JSRuntimeError: <complicated error message>
It's not important whether the request is _ever_ sent; the CORS
exercise requires sending it but the standard implementation does not
send it.
Testing SameSite cookies and CSRF
=================================
`SameSite` cookies should be sent on cross-site `GET`s and
same-site `POST`s but not on cross-site `POST`s.
Cookie without `SameSite` have already been tested above. Let's create
a `SameSite` cookie to start.
>>> url = "http://test.test.chapter10/"
>>> test.socket.respond(url, b"HTTP/1.0 200 OK\r\nSet-Cookie: bar=baz; SameSite=Lax\r\n\r\nempty")
>>> tab.load(url)
>>> browser.COOKIE_JAR["test.test.chapter10"]
('bar=baz', {'samesite': 'lax'})
Now the browser should have `bar=baz` as a `SameSite` cookie for
`test.test.chapter10`. First, let's check that it's sent in a same-site `GET`
request:
>>> url2 = "http://test.test.chapter10/2"
>>> test.socket.respond(url2, b"HTTP/1.0 200 OK\r\n\r\n2")
>>> tab.load(url2)
>>> b'cookie: bar=baz' in test.socket.last_request(url2).lower()
True
Now let's submit a same-site `POST` and check that it's also sent
there:
>>> url3 = "http://test.test.chapter10/add"
>>> test.socket.respond(url3, b"HTTP/1.0 200 OK\r\n\r\nAdded!", method="POST")
>>> tab.load(url3, body="who=me")
>>> req = test.socket.last_request(url3).lower()
>>> req.startswith(b'post')
True
>>> b'cookie: bar=baz' in req
True
>>> b'content-length: 6' in req
True
>>> req.endswith(b'who=me')
True
Now we navigate to another site, navigate back by `GET`, and the
cookie should *still* be sent:
>>> url4 = "http://other.site.chapter10/"
>>> test.socket.respond(url4, b"HTTP/1.0 200 OK\r\n\r\nHi!")
>>> tab.load(url4)
>>> tab.load(url)
>>> b'cookie: bar=baz' in test.socket.last_request(url).lower()
True
Finally, let's try a cross-site `POST` request and check that in this
case the cookie is *not* sent:
>>> tab.load(url4)
>>> tab.load(url3, body="who=me")
>>> req = test.socket.last_request(url3).lower()
>>> req.startswith(b'post')
True
>>> b'content-length: 6' in req
True
>>> req.endswith(b'who=me')
True
Testing Content-Security-Policy
===============================
We test `Content-Security-Policy` by checking that subresources are
loaded / not loaded as required. To do that we need a page with a lot
of subresources:
>>> url = "http://test.test.chapter10/"
>>> body = """<!doctype html>
... <link rel=stylesheet href=http://test.test.chapter10/css />
... <script src=http://test.test.chapter10/js></script>
... <link rel=stylesheet href=http://library.test.chapter10/css />
... <script src=http://library.test.chapter10/js></script>
... <link rel=stylesheet href=http://other.test.chapter10/css />
... <script src=http://other.test.chapter10/js></script>
... """
>>> test.socket.respond(url, b"HTTP/1.0 200 OK\r\n\r\n" + body.encode("utf8"))
We also need to create all those subresources:
>>> test.socket.respond_ok(url + "css", "")
>>> test.socket.respond_ok(url + "js", "")
>>> url2 = "http://library.test.chapter10/"
>>> test.socket.respond_ok(url2 + "css", "")
>>> test.socket.respond_ok(url2 + "js", "")
>>> url3 = "http://other.test.chapter10/"
>>> test.socket.respond_ok(url3 + "css", "")
>>> test.socket.respond_ok(url3 + "js", "")
Now with all of these URLs set up, let's load the page without CSP and
check that all of these requests were made:
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
Script returned: None
Script returned: None
Script returned: None
>>> [test.socket.made_request(url + "css"),
... test.socket.made_request(url + "js")]
[True, True]
>>> [test.socket.made_request(url2 + "css"),
... test.socket.made_request(url2 + "js")]
[True, True]
>>> [test.socket.made_request(url3 + "css"),
... test.socket.made_request(url3 + "js")]
[True, True]
Now let's reload the page, but with CSP enabled for `test.test.chapter10` and
`library.test.chapter10` but not `other.test.chapter10`:
>>> test.socket.clear_history()
>>> test.socket.respond(url, b"HTTP/1.0 200 OK\r\n" + \
... b"Content-Security-Policy: default-src http://test.test.chapter10 http://library.test.chapter10\r\n\r\n" + \
... body.encode("utf8"))
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
Script returned: None
Script returned: None
Blocked script http://other.test.chapter10/js due to CSP
Blocked style http://other.test.chapter10/css due to CSP
The URLs on `test.test.chapter10` and `library.test.chapter10` should have been loaded:
>>> [test.socket.made_request(url + "css"),
... test.socket.made_request(url + "js")]
[True, True]
>>> [test.socket.made_request(url2 + "css"),
... test.socket.made_request(url2 + "js")]
[True, True]
However, neither script nor style from `other.test.chapter10` should be loaded:
>>> [test.socket.made_request(url3 + "css"),
... test.socket.made_request(url3 + "js")]
[False, False]
Let's also test that XHR is blocked by CSP. This requires a little
trickery, because cross-site XHR is already blocked, so we need a CSP
that restricts all sites---but then we can't load and run any
JavaScript!
>>> url = "http://weird.test.chapter10/"
>>> test.socket.respond(url, b"HTTP/1.0 200 OK\r\n" + \
... b"Content-Security-Policy: default-src\r\n\r\nempty")
>>> this_browser.load(url)
>>> tab = this_browser.tabs[-1]
>>> tab.js.run("""
... x = new XMLHttpRequest()
... x.open('GET', 'http://weird.test.chapter10/xhr', false);
... x.send();""") #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
_dukpy.JSRuntimeError: <complicated wrapper around 'Cross-origin XHR blocked by CSP'>
Tests for WBE Chapter 10 Exercise `Certificate errors`
============================================
Description
-----------
When accessing an HTTPS page, the web server can send an invalid certificate
(badssl.com hosts various invalid certificates you can use for testing).
In this case, the wrap_socket function will raise a certificate error;
Catch these errors and show a warning message to the user.
For all other HTTPS pages draw a padlock (spelled \N{lock}) in the address bar.
Extra Requirements
------------------
* If you first connect your socket then wrap it the exception will be thrown when
calling wrap_socket. If you wrap the socket before calling connect then the
exception is thrown when calling connect.
* For the warning message to the user simply do not load the page and instead
display the following webpage:
```html
<!doctype html>
Secure Connection Failed
```
Test code
---------
Boilerplate.
>>> import test
>>> _ = test.socket.patch().start()
>>> _ = test.ssl.patch().start()
>>> test.NO_CACHE = True
>>> test.NORMALIZE_FONT = True
>>> import browser
Check that using http does not add the lock character.
>>> test.TK_CANVAS_CALLS = list()
>>> url = "http://test.test.chapter10-certificate-errors/"
>>> test.socket.respond_ok(url, "Insecure page")
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
>>> tk_text = [c for c in test.TK_CANVAS_CALLS if
... c.startswith("create_text")]
>>> any("\N{lock}" in c for c in tk_text)
False
The lock character should be displayed when the page is https and no errors
occur.
>>> test.TK_CANVAS_CALLS = list()
>>> url = "https://test.test.chapter10-certificate-errors/"
>>> test.socket.respond_ok(url, "Secure page")
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
>>> tk_text = [c for c in test.TK_CANVAS_CALLS if
... c.startswith("create_text")]
>>> any("\N{lock}" in c for c in tk_text)
True
When the certificate is invalid display the above page and do not display a
lock.
>>> test.TK_CANVAS_CALLS = list()
>>> this_browser = browser.Browser()
>>> this_browser.load("https://untrusted-root.badssl.com/")
>>> browser.print_tree(this_browser.tabs[0].document)
DocumentLayout()
BlockLayout(x=13, y=18, width=774, height=15.0)
InlineLayout(x=13, y=18, width=774, height=15.0)
LineLayout(x=13, y=18, width=774, height=15.0)
TextLayout(x=13, y=20.25, width=72, height=12, font=Font size=12 weight=normal slant=roman style=None)
TextLayout(x=97, y=20.25, width=120, height=12, font=Font size=12 weight=normal slant=roman style=None)
TextLayout(x=229, y=20.25, width=72, height=12, font=Font size=12 weight=normal slant=roman style=None)
>>> tk_text = [c for c in test.TK_CANVAS_CALLS if
... c.startswith("create_text")]
>>> any("\N{lock}" in c for c in tk_text)
False
Tests for WBE Chapter 10 Exercise `New inputs`
============================================
Description
-----------
Add support for hidden and password input elements.
Hidden inputs shouldn’t show up or take up space, while password input elements
should show ther contents as stars instead of characters.
Extra Requirements
------------------
* To hide the input element set its width and height to `0.0`
Test code
---------
Boilerplate.
>>> import test
>>> _ = test.socket.patch().start()
>>> _ = test.ssl.patch().start()
>>> test.NO_CACHE = True
>>> test.NORMALIZE_FONT = True
>>> import browser
Make a page with a hidden input element.
>>> url = "http://test.test.chapter10-new-inputs/"
>>> page = """<!doctype html>
... <form action="/tricky" method=POST>
... <p>Not hidden: <input name=visible value=1></p>
... <p>Hidden: <input type=hidden name=invisible value=doNotShowMe></p>
... <p><button>Submit!</button></p>
... </form>"""
>>> test.socket.respond_ok(url, page)
>>> test.socket.respond(url + "tricky", b"HTTP/1.0 200 OK\r\n\r\nEmpty", "POST")
The hidden element should not show up.
There are many ways to achieve this effect, we will set the width and height of
the element to 0.0.
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
>>> browser.print_tree(this_browser.tabs[0].document)
DocumentLayout()
BlockLayout(x=13, y=18, width=774, height=45.0)
BlockLayout(x=13, y=18, width=774, height=45.0)
BlockLayout(x=13, y=18, width=774, height=45.0)
InlineLayout(x=13, y=18, width=774, height=15.0)
LineLayout(x=13, y=18, width=774, height=15.0)
TextLayout(x=13, y=20.25, width=36, height=12, font=Font size=12 weight=normal slant=roman style=None)
TextLayout(x=61, y=20.25, width=84, height=12, font=Font size=12 weight=normal slant=roman style=None)
InputLayout(x=157, y=20.25, width=200, height=12)
InlineLayout(x=13, y=33.0, width=774, height=15.0)
LineLayout(x=13, y=33.0, width=774, height=15.0)
TextLayout(x=13, y=35.25, width=84, height=12, font=Font size=12 weight=normal slant=roman style=None)
InputLayout(x=109, y=35.25, width=0.0, height=0.0)
InlineLayout(x=13, y=48.0, width=774, height=15.0)
LineLayout(x=13, y=48.0, width=774, height=15.0)
InputLayout(x=13, y=50.25, width=200, height=12)
Submission of the form should still pass along the value.
>>> this_browser.handle_click(test.Event(21, 100+58))
>>> req = test.socket.last_request(url + "tricky").decode().lower()
>>> req.startswith("post")
True
>>> "content-length: 31" in req
True
>>> req.endswith("visible=1&invisible=donotshowme")
True
Make a page with a password input element.
>>> url = "http://test.test.chapter10-new-inputs/"
>>> page = """<!doctype html>
... <form action="/login" method=POST>
... <p>Name: <input name=name value=Skroob></p>
... <p>Password: <input type=password name=password value=12345></p>
... <p><button>Submit!</button></p>
... </form>"""
>>> test.socket.respond_ok(url, page)
>>> test.socket.respond(url + "login", b"HTTP/1.0 200 OK\r\n\r\nEmpty", "POST")
The password element should be all `*`.
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
>>> browser.print_tree(this_browser.tabs[0].document)
DocumentLayout()
BlockLayout(x=13, y=18, width=774, height=45.0)
BlockLayout(x=13, y=18, width=774, height=45.0)
BlockLayout(x=13, y=18, width=774, height=45.0)
InlineLayout(x=13, y=18, width=774, height=15.0)
LineLayout(x=13, y=18, width=774, height=15.0)
TextLayout(x=13, y=20.25, width=60, height=12, font=Font size=12 weight=normal slant=roman style=None)
InputLayout(x=85, y=20.25, width=200, height=12)
InlineLayout(x=13, y=33.0, width=774, height=15.0)
LineLayout(x=13, y=33.0, width=774, height=15.0)
TextLayout(x=13, y=35.25, width=108, height=12, font=Font size=12 weight=normal slant=roman style=None)
InputLayout(x=133, y=35.25, width=200, height=12)
InlineLayout(x=13, y=48.0, width=774, height=15.0)
LineLayout(x=13, y=48.0, width=774, height=15.0)
InputLayout(x=13, y=50.25, width=200, height=12)
>>> form = this_browser.tabs[0].document.children[0].children[0].children[0]
>>> para = form.children[1].children[0]
>>> pswd = para.children[1]
>>> dl = list()
>>> pswd.paint(dl)
>>> dl #doctest: +NORMALIZE_WHITESPACE
[DrawRect(top=35.25 left=133 bottom=47.25 right=333 color=lightblue),
DrawText(top=35.25 left=133 bottom=47.25 text=***** font=Font size=12 weight=normal slant=roman style=None)]
Submission of the form should still pass along the value.
>>> this_browser.handle_click(test.Event(21, 100+58))
>>> req = test.socket.last_request(url + "login").decode().lower()
>>> req.startswith("post")
True
>>> "content-length: 26" in req
True
>>> req.endswith("name=skroob&password=12345")
True
Tests for WBE Chapter 10 Exercise `Referer`
============================================
Description
-----------
When your browser visits a web page, or when it loads a CSS or JavaScript file,
it sends a Referer header (Yep, spelled that way.) containing the URL it is
coming from.
Sites often use this for analytics.
Implement this in your browser.
However, some URLs contain personal data that they don’t want revealed to other
websites, so browsers support a Referrer-Policy header, which can contain
values like no-referer (never send the Referer header when leaving this
page) or same-origin (only do so if navigating to another page on the same
origin). Implement those two values for Referer-Policy.
Extra Requirements
------------------
* Note the difference in spelling, the headers are `Referer` and
`Referrer-Policy`.
Test code
---------
Boilerplate.
>>> import test
>>> _ = test.socket.patch().start()
>>> _ = test.ssl.patch().start()
>>> test.NO_CACHE = True
>>> test.NORMALIZE_FONT = True
>>> import browser
Load a page with some CSS, and check that `Referer` is used.
>>> url = "http://test.test.chapter10-referer-1/"
>>> body = """<!DOCTYPE html>
... <link rel="stylesheet" href="style.css" />
... Empty"""
>>> test.socket.respond_ok(url, body)
>>> test.socket.respond_ok(url + "style.css", "")
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
>>> req = test.socket.last_request(url + "style.css").decode().lower()
>>> "referer:" in req
True
>>> "referer: {}".format(url) in req
True
Now load a page setting the `Referrer-Policy` header to `no-referrer`, and
check that `Referer` is not used.
>>> url = "http://test.test.chapter10-referer-2/"
>>> body = """<!DOCTYPE html>
... <script src=http://test.test.chapter10-referer-2/same.js></script>
... <script src=http://test.diff.chapter10-referer-2/diff.js></script>
... Empty"""
>>> body = b"HTTP/1.0 200 OK\r\nReferrer-Policy: no-referrer\r\n\r\n" + body.encode("utf8")
>>> test.socket.respond(url, body)
>>> test.socket.respond_ok("http://test.test.chapter10-referer-2/same.js", "")
>>> test.socket.respond_ok("http://test.diff.chapter10-referer-2/diff.js", "")
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
Script returned: None
Script returned: None
>>> req = test.socket.last_request("http://test.test.chapter10-referer-2/same.js").decode().lower()
>>> "referer:" in req
False
>>> req = test.socket.last_request("http://test.diff.chapter10-referer-2/diff.js").decode().lower()
>>> "referer:" in req
False
Finally load a page setting the `Referrer-Policy` header to `same-origin`, and
check that `Referer` is only used when the origin is the same.
>>> url = "http://test.test.chapter10-referer-3/"
>>> body = """<!DOCTYPE html>
... <script src=http://test.test.chapter10-referer-3/same.js></script>
... <script src=http://test.diff.chapter10-referer-3/diff.js></script>
... Empty"""
>>> body = b"HTTP/1.0 200 OK\r\nReferrer-Policy: same-origin\r\n\r\n" + body.encode("utf8")
>>> test.socket.respond(url, body)
>>> test.socket.respond_ok("http://test.test.chapter10-referer-3/same.js", "")
>>> test.socket.respond_ok("http://test.diff.chapter10-referer-3/diff.js", "")
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
Script returned: None
Script returned: None
>>> req = test.socket.last_request("http://test.test.chapter10-referer-3/same.js").decode().lower()
>>> "referer:" in req
True
>>> "referer: {}".format(url) in req
True
>>> req = test.socket.last_request("http://test.diff.chapter10-referer-3/diff.js").decode().lower()
>>> "referer:" in req
False
Tests for WBE Chapter 10 Exercise `Script access`
============================================
Description
-----------
Implement the document.cookie JavaScript API.
Reading this field should return a string containing the cookie value ~~and
parameters~~, formatted similarly to the Cookie header.
Writing to this field updates the cookie value and parameters, just like
receiving a Set-Cookie header does.
Also implement the HttpOnly cookie parameter; cookies with this parameter
cannot be read or written from JavaScript.
Extra Requirements
------------------
* Implementation is made easier since the browser supports one cookie per host,
so getting will return either 0 or 1 cookie and setting will overwrite the
cookie (if allowable).
Test code
---------
Boilerplate.
>>> import test
>>> _ = test.socket.patch().start()
>>> _ = test.ssl.patch().start()
>>> test.NO_CACHE = True
>>> test.NORMALIZE_FONT = True
>>> import browser
Open an empty page to get the javascript instance
>>> url = "http://test.test.chapter10-script-access/"
>>> test.socket.respond_ok(url, "Empty")
>>> this_browser = browser.Browser()
>>> this_browser.load(url)
>>> js = this_browser.tabs[0].js
Check that javascript doesn't see any cookies, and that the cookie jar is empty.
>>> js.run("console.log(document.cookie)")