Measuring QUIC and TLS censorship with OONI Probe

About Me

Kathrin Elmenhorst, final year B.Sc. CS student, from Osnabrueck, Germany.
I am interested in digital rights and tech education.

About My Project

I will be working with the team at the Open Observatory of Network Interference (OONI) to develop network experiments measuring the blocking of websites via TLS and QUIC. I will also work on the further integration of my previous contribution which is an HTTP/3 (HTTP over QUIC) extension for the OONI engine.
GSoC project

My Time Zone

Central European Time (UTC+2)

Getting in Touch

Please message me if you are interested in my project or want to chat about all things Internet and society.

May 24, 2021 - June 4, 2021: Community Bonding

  • Me and my fellow GSoC student at OONI were welcomed by the team in an introductory meeting. We talked about the modes of agile operation at OONI and got introduced to the used organizational tools, such as the Zenhub extension for issue organization. We agreed on the level and ways of our participation in the team and planned for us students to take part in the bi-weekly sprint meetings.
  • I am following and participating in various OONI Slack channels, so I know what is going on in the community.
  • I had a meeting with my mentor where we talked about the specifics of my project and planned the first steps of the coding phase.
  • I wrote and submitted the issue for my first week of coding so I am ready to start off directly.
1 Like

June 7, 2021 - June 11, 2021: Start of the coding period

  • I participated in the bi-weekly sprint planning meeting. I learned how these meetings work and that tasks are not only defined for each team member, but can also be labeled as “team priority” for issues that are prioritized to be completed during this sprint and require team collaboration.
  • I started working on a refactoring project regarding the mapping of network errors to OONI censorship-related error strings. The goal of this refactoring is to dissolve packet dependencies and localize the error wrapping. The refactoring touched many parts of the codebase and allowed me to get familiar again with the OONI engine, and I already learned more about Golang coding.
  • The refactoring will hopefully completed next week. Afterwards I want to start researching more about TLS parroting.

June 14 - June 18

What I did:

Finished the refactoring for error classification.
Outcomes:

  • improved handling of QUIC errors, by
    a) importing QUIC error codes from quic-go
    b) importing TLS alert codes from the RFC
    c) generally stop relying on possibly changing error strings
  • improved handling of system errors: introduced system errno constants to detect and identify system errors
    → this solved the problem of detecting localized windows error strings (issue 1526)
  • divided the classifier code (that translates errors into OONI error strings) by layers, so that each of the following protocol layers has it‘s own classifier with layer-specific errors
    ⁃ QUIC
    ⁃ TLS over TCP
    ⁃ resolve
    A general classifier remains to handle errors that can occur on multiple layers.
    → this change helps with the maintainability of the classification code and improves the interface of the errorx package and therefore fulfills the requirements defined in issue 1505.

Challenges:

  • During this week, with my mentor I found out, that the original refactoring design of introducing layer/operation specific error types will not work with the current architecture of OONI Probe. This is, among other reasons, due to the fact that potentially occurring errors can not be sharply separated between operations. Furthermore, the current codebase relies on the classification results in many places, more than expected, so it became evident that having a single error wrapper type, with a field defining the failed operation, is the best solution for the time being.
    → the planned error wrapping refactoring therefore became an error classification refactoring: the localization of layer-specific errors as originally planned was not realized by introducing operation-specific error wrappers, but by subdividing the classification code used by the single error wrapper.
  • This week, I struggled with structuring my work time and planning the immediate To Do‘s. Although the outcome of this week‘s work was met the requirements and original plan, I want to be more intentional and organized when it comes to my own time and task management. I communicated this with my mentor and we agreed to schedule a weekly Monday meeting to layout the week ahead and discuss the extent and time frame of upcoming tasks.

Next week:

  • I will start tackling the TLS parroting issue: After continuing my research on this topic I want to have a clear design idea for implementing TLS parroting in OONI Probe at the end of the next week.
  • I will stay in close communication with my mentor, exchange ideas and get feedback about my implementation ideas.
  • In terms of deliverables, this means: I will do proof of concept tests, and provide a graphic draft of the design.
1 Like

June 21 - June 25

What I did:

  • research*: TLS fingerprints
    • characteristic structure of the TLS ClientHello (or ServerHello) due to the used implementation on the client/server machine
    • ClientHello is not encrypted
    • presence/absence, length and order, as well as data of connection negotiation parameters, such as supported ciphers, algorithms, and extensions
  • research*: Fingerprinting censorship methods
    • used to block circumvention tools (such as Tor), or other unwanted clients; or to identify blocked websites (website fingerprinting)
    • two main implementation approaches: blacklisting and whitelisting
  • research*: TLS parroting (client side)
    • evasion of fingerprint blocking by mimicking the ClientHello fingerprint of popular client implementations
    • example: the ClientHello of a censorship circumvention tool is made to look like Google Chrome’s ClientHello and can therefore not be identified by the censor. In order to block the tool, the censor has to accept a lot of collateral damage, if the parroted implementation is popular enough
    • challenge: keeping up with browser updates (!)
    • alternative to mimicking: ClientHello randomizing, which also works against blacklisting censors but can be filtered out by the censor without causing collateral damage
  • implementation: first draft of enabling OONI Probe to use uTLS, the golang library for TLS parroting, with some tests
  • implementation: UTLSHandshaker that configures OONI’s ClientHello so that it looks like the ClientHello of Google Chrome 83, with tests

Challenges:

  • uTLS can only be used with golang/net http2.Transport, OONI Probe uses golang/net http.Transport

    • solution: roundTripper wrapper which performs a TCP and TLS handshake and determines based on the ALPN which transport type to use (see figure)
    • this change affects many probe experiments, so it is not trivial to ensure compatibility with the rest of the codebase → testing is very important

    roundtripper

Retrospective

  • TLS parroting research :heavy_check_mark:
  • design idea for implementing TLS parroting in OONI Probe :heavy_check_mark:
  • close communication with my mentor, quick feedback cycles: Monday meeting :heavy_check_mark:, I think I can still improve my communication and update frequency :heavy_minus_sign:
  • proof of concept tests → first draft :heavy_check_mark:
  • graphic draft of the design → roundTripper wrapper :heavy_check_mark:

Next week:

  • write more tests for the roundTripper wrapper
  • improve code design and style
  • submit PR with the changes
  • capture OONI Probe’s fingerprint when parroting, and use tlsfingerprint.io/pcap to analyze it
    • aim: make sure that the fingerprint we are using is popular and up-to-date

* research resources: Frolov, S., & Wustrow, E. (2019). The use of TLS in Censorship Circumvention. In NDSS. )

1 Like

June 28 - July 2

What I did:

  • PR uTLS for OONI: I wrote comprehensive unit tests for the implementation I did last week. Also, I improved the code in terms of style. My mentor reviewed the code and I implemented the suggested design changes which included making the uTLS integration part of the new netxlite package. Thus, the implementation is now in line with the current refactoring efforts.

  • I used Wireshark and tlsfingerprint.io/pcap to analyze the popularity of the fingerprint issued by OONI’s TLS ClientHello:
    Normal OONI TLS ClientHello:
    Screenshot from 2021-07-02 11-25-55
    Mimicking Google Chrome ClientHello:
    Screenshot from 2021-07-02 11-26-12

  • The uTLS integrating part was merged into master.

Challenges:

  • Discussing the roundTripper with my mentor, we realized that it would be more useful to have a more modular approach, making it possible to separately perform operations like the DNS lookup, the TCP handshake and the TLS handshake. As roundTripper wraps all of those things together it cannot be used in such a modular way.
    • I drafted up a plan on how we can achieve modular testing of separate operations.

Retrospective:

  • wrote more tests for the roundTripper :heavy_check_mark:
  • improve code design and style :heavy_check_mark:
  • submit PR with the changes :heavy_check_mark:
  • capture OONI Probe’s fingerprint when parroting, and use tlsfingerprint.io/pcap to analyze it :heavy_check_mark:
    • aim: make sure that the fingerprint we are using is popular and up-to-date :heavy_check_mark:
      → this should be done frequently, to find out the best current fingerprint to parrot

Next week:

  • Implement a draft of a new webconnectivity version following the plan I laid. Discuss this with my mentor to find out what the advantages and disadvantages of this version would be in comparison to the current webconnectivity design.

July 5 - July 9

What I did

  • I re-implemented OONI’s Web Connectivity experiment following the draft developed last week. (branch issue/1702)
    • The new design is modular, which allows us to have more control over the different operations during the connection to a host.
  • This new Web Connectivity combines the two main objectives of my GSoC project:
    • Enable OONI to use TLS parroting for measuring: The new modular workflow of Web Connectivity allows to choose the appropriate HTTP transport type (see Challenges of entry June 21 - June 25) for the negotiated protocol. This decision is necessary to use the uTLS library for TLS parroting.
    • Integrate QUIC transport (HTTP/3) in Web Connectivity: Part of the new implementation is a QUIC measurement step. The aim is to let Web Connectivity use the test helper - which runs on OONI servers - to determine whether the target host supports QUIC.

Challenges

  • The new experiment has to manually handle HTTP redirects.

Retrospective:

  • Implement a draft of a new Web Connectivity version following the plan I laid. Discuss this with my mentor. :heavy_check_mark:

Next week

  • I will continue working on HTTP redirects in the new design.
  • I want to finish integrating the saving of measurement data for the individual measuring operations.
  • I want to figure out how to use cookies in the new design, and implement an appropriate solution.
1 Like

July 12 - July 16

What I did

  • I researched HTTP cookie and redirection mechanisms and implemented the new version of the Web Connectivity experiment accordingly.
    • In accordance with Golang’s http.Client, the experiment Measurer uses the same HTTP cookie jar for all redirects.
    • When following redirects, the Measurer will use the same headers.
    • It stops after 10 consecutive redirects.
    • Optimization: Only one redirect request is followed, assuming that every endpoint that belongs to the domain responds with the same redirect location. This assumption is necessary so that the number of requests does not exponentially grow with every redirect.
    • Distinction between {301,302,303} and {307,308} redirects. The latter status codes guarantee that the method and the body will not be changed when the redirected request is made.
  • I used the archival package to save measurement data for all currently implemented operations in the new Web Connectivity experiment. I compared the report files of the old and new versions of Web Connectivity to make sure that the saved data is ~complete. Along with adding the control measurement, I will make sure to save control measurement data next week.
  • I modified the DNS resolver to translate domain names to ASCII so that international domain names are supported by Web Connectivity.
  • I enabled the new Web Connectivity experiment to use uTLS for TLS fingerprint parroting.
    • The used client for parroting (e.g. “Chrome”) can be configured from the command line of over the Config interface.
  • I wrote unit and integration tests for the above described features and modifications.

The current state of the work can be found in branch issue/1702.

Challenges

  • The new version of Web Connectivity has a longer per-domain runtime than the old version, because
    a) a chain of IP addresses is measured for each domain, as opposed to following only one address,
    b) the new version also measures HTTP/3 (over QUIC). This additional measurement not only adds a QUIC handshake and HTTP/3 roundtrip, the underlying QUIC implementation seems to do a couple of retries when a host does not support QUIC which cause extra delay. This problem will be avoided by implementing a QUIC check in the OONI test helper that indicates whether the QUIC measurement should happen or not (see “Next week”).
  • I struggled to figure out in how far we can save “network_events”, such as Read/Write operations. It is possible that we need a Saver for this, which is a data structure used by the old version of Web Connectivity. I will discuss this issue with my mentor.

Retrospective

  • continue working on HTTP redirects in the new design :heavy_check_mark:
  • finish integrating the saving of measurement data for the individual measuring operations :heavy_check_mark: (except for the control measurement data)
  • figure out how to use cookies in the new design, and implement an appropriate solution :heavy_check_mark:
  • extra: use uTLS fingerprints for TLS parroting :heavy_check_mark:

Next week

  • Add control measurements to the new version of Web Connectivity. Control measurements are handled by the OONI test helper which runs the same measurements from an uncensored network. I will make sure to talk to my mentor about how the test helper works, if unclear.
  • Save control measurement data.
  • Avoid always testing QUIC: I will modify the test helper so that it tests whether a host supports QUIC, and provide this information to the client. According to the control, the client should do the HTTP/3 (over QUIC) measurement, or not.
  • Continue writing unit and integration tests.
  • Possibly: Let test helper do a DNS lookup, and unite the results of the client lookup and the test helper lookup. This ensures that we can measure although the DNS lookup failed.
1 Like

July 19 - July 23

What I did

  • Added control measurements to the new version of Web Connectivity.
    • I used the current web connectivity’s control client as a template.
    • The OONI testhelper performs the measurement from an uncensored network to have a reference value for the target host. This is necessary to distinguish permanent or temporary outages on the server side from censorship.
  • United the results of the client lookup and the test helper lookup.
    • With this addition, it is possible to perform the measurement although the DNS lookup on the client failed. It helps improve the information gain of web connectivity in networks with DNS blocking.
  • Added a saver to log control measurement data.
    • QUIC handshakes get a specific tag, so they can be distinguished from regular TLS handshakes.
    • TODO: annotate HTTP measurement to indicate whether HTTPS or HTTP/3 was used
  • Added QUIC support check.
    • Originally, the plan was to modify the test helper so that it performs an additional test of whether a host supports QUIC, and provide this information to the client. It turned out that we already have this information, without any need to modify the testhelper implementation. Orientating towards browser behavior, the OONI client inspects the “Alt-Svc” header field of the HTTP request result from the control measurement.
    • Hosts use the “Alt-Svc” header to announce whether they support QUIC.
    • Using the information from the control measurement, we can find out whether hosts support HTTP/3 and run QUIC/HTTP/3 measurements even in networks where HTTPS traffic is blocked.
  • Wrote unit and integration tests.
  • Adapted the redirection limit to current browser implementations which use a limit of 20 redirects. (The web connectivity client usually aims to behave similarly to browsers.)
  • I compared the runtime of the current and the new version of Web Connectivity when measuring 1475 entries of the default URL test list.
    • Disclaimer: I am aware that this comparison is not free of bias and that there might be factors that influenced the absolute runtimes of the experiments. This is why I will not mention the absolute duration measurements.
    • Although the new Web Connectivity measures all resolved IP addresses (instead of only one) and additionally does a QUIC/HTTP/3 measurement for every endpoint, the runtime was nearly the same. This is probably due to the fact that a) we use parallelism to measure the multiple IP endpoints per host, and b) the new version has a lot less complexity.
  • I compared the data format generated by the current and the new version of Web Connectivity and made sure that we measure the same (and more) pieces of information as before.
  • Added a follow up measurement that Web Connectivity performs in case a TLS handshake fails: The same TLS handshake is repeated using example.com as the Server Name Indication (SNI).
    • If the TLS handshake with the example SNI succeeds we have an indication for a SNI based censorship method which uses the clear text SNI in the Client Hello to filter blocklisted websites.

Challenges

Honestly, it was pretty smooth sailing this week. :sailboat: :sun_with_face:

Retrospective

  • Add control measurements to the new version of Web Connectivity. :heavy_check_mark:
  • Save control measurement data. :heavy_check_mark:
  • Avoid always testing QUIC: Modify the test helper so that it tests whether a host supports QUIC, and provide this information to the client. → other solution, see above :heavy_check_mark:
  • Continue writing unit and integration tests. :heavy_check_mark: but more are required :heavy_minus_sign:
  • Possibly: Let test helper do a DNS lookup, and unite the results of the client lookup and the test helper lookup. :heavy_check_mark:

Next week

  • Have a comprehensive check-in about the project with my mentor and discuss the next steps.
  • Finish writing unit and integration tests.
    • Test the SNI follow up experiment.
    • Generate specific network failures and verify that the experiment leads to the expected result.
  • Measure with an input list of known to be QUIC supporting hosts, to evaluate how many hosts fail to announce their QUIC support via the HTTP Alt-Svc header.
  • Test QUIC download time under low bandwidth conditions using netem.
1 Like

July 26 - July 30

What I did

  • I had a long and comprehensive meeting with my mentor, evaluating the current experiment and the tasks that are left to do.
  • I modified the new web connectivity testhelper so that it performs an HTTP/3 control measurement. The OONI test helper runs the same measurements as the probe but from an uncensored network, to have a reference for what to expect when requesting a certain host.
  • I restructured and debugged the redirection mechanism of the experiment. The new code manually redirects only when the redirect targets an external connection, i.e. when the Golang HTTP client has to establish a new TCP connection. For cases where the redirect targets an internal resource we make use of the client’s strategy to reuse the same underlying TCP connection w/o connecting again.
    • Problem: this is probably not applicable to HTTP/3 (TODO: investigate the quic-go/http3 implementation more and find out exactly what happens)
  • I fixed inconsistencies in terms of HTTP cookie handling
    • e.g.: use a separate cookie jar for TCP and QUIC based HTTP
    • e.g.: use a separate cookie jar for each endpoint

Challenges

  • This week’s tasks turned out to be very different from what I had expected.
  • Seemingly, my work this week has uncovered more problems than it has solved.
  • I could not move forward with the low bandwidth testing, nor the testing of the SNI follow-up experiment.

Retrospective

  • Have a comprehensive check-in about the project with my mentor and discuss the next steps. :x:
  • Finish writing unit and integration tests :x:
  • Measure with an input list of known to be QUIC supporting hosts, to evaluate how many hosts fail to announce their QUIC support via the HTTP Alt-Svc header. :x:
  • Test QUIC download time under low bandwidth conditions using netem. :x:

Next week

  • Catch up on the testing plans of last week.
  • Review round with my mentor of the experiment code with the goal to merge it into the master branch piece by piece.

August 02 - August 06

What I did:

  • I implemented the new version of the Web Connectivity Test Helper following the specification defined by my mentor. The specification is a product of our considerations over the last few weeks.
  • The new design supports testing multiple endpoints for a URL, as well as HTTP/3 connectivity.
  • I created a pull request for the new version of the Web Connectivity Test Helper:

    This pull request adds a prototype of the new web connectivity test helper which supports testing multiple IP endpoints per domain and introduces HTTP/3 control measurements.

  • Scheme of the new testhelper code design:

Retrospective

  • Catch up on the testing plans of last week.
    • :heavy_check_mark: for the test helper code which we focused on this week
    • :x: for the experiment (nwebconnectivity) code which I will handle next week
  • Review round with my mentor of the experiment code with the goal to merge it into the master branch piece by piece. :heavy_check_mark:

Next week

  • Finalize the merge of the test helper code.
  • Apply design of the test helper to nwebconnectivity.
  • Create pull request for the nwebconnectivity code, and merge it.

August 09 - August 13

What I did:

  • We did another iteration of the testhelper redesign, building upon the previous code.
  • I reviewed and contributed to the specification for the testhelper design.
  • I implemented the new design in the current pull request.
  • I drafted a prototype of the corresponding new Web Connectivity experiment. This experiment will use the testhelper’s control measurement to know what URLs to measure. This is a new approach which allows us to avoid dilemmas and potential error sources around HTTP redirection and cookies. The pull request can be found here.

Retrospective

  • Finalize the test helper code. :heavy_check_mark:
  • Apply design of the test helper to nwebconnectivity. :heavy_check_mark:
  • Create pull request for the nwebconnectivity code, and merge it. :x:

Next week

  • Documentation and finishing touches.

Google Summer of Code 2021 at OONI deliverable

This is the summary of the outcome of my GSoC experience, including links to the code and explanations: