Colaboratory, or Colab, is a hosted Jupyter notebook service requiring zero setup and providing free access to compute resources. It is a convenient and powerful way to share research, and we use it extensively in The Lab.
What’s The Lab?
The Lab is the RW Pro group’s portal for doing collaborative research together as a community. The key goals are:
- To develop members’ quant research skills by getting them hands-on with the research process.
- To scale the research effort through community collaboration.
- To make the fruits of that research effort available to the entire community.
In short, we grow people’s research skills and the number of edges available to trade faster than an individual could grow them alone.
Connecting Colab to a local Jupyter runtime
Colab allows you to create, run, and share Jupyter notebooks without having to download or install anything. Integration with GitHub means that you can work entirely in the cloud:
While working in the cloud has benefits – such as no local setup – there are also limitations. For example:
- Repeatedly setting up the research environment from scratch
- Session disconnection if idle for too long
- Memory limitations
To get around such constraints, you might consider connecting Colab to a Jupyter server running locally.
But why run Colab locally, rather than working in a vanilla Jupyter notebook?
Mainly for the sharing and collaboration tools: one-click loading and pushing to GitHub, easy access to cloud storage buckets, and “snippet notebooks.”
Snippet notebooks contain chunks of self-contained code that can be injected into a working notebook. This is a great way to collaborate, ensure consistency and reproducibility, increase productivity, and share tools such that the code is visible in the notebook. We find this suits collaborative quant research more than say abstracting such functionality away in a package:
How to connect Colab to a local Jupyter runtime
Step 1: Install Jupyter
The easiest way is via Conda:
conda install -c conda-forge jupyterlab
or pip:
pip install jupyterlab
More detailed instructions can be found here.
Step2: Install Jupyter server extension for using a WebSocket to proxy HTTP traffic
The colab team authored thejupyter_http_over_ws extension. Install it by doing:
pip install jupyter_http_over_ws
Then enable it by doing:
jupyter serverextension enable --py jupyter_http_over_ws
You can read more detailed information about this extension on its GitHub repository.
Step 3: Start a local Jupyter server
We need a local Jupyter server that trusts WebSocket connections from the Colab frontend. The following command and flags accomplish this:
jupyter notebook \
--NotebookApp.allow_origin='https://colab.research.google.com' \
--port=8888 \
--NotebookApp.port_retries=0
Once the server has started, it will print a message with the initial backend URL used for authentication. You’ll need a copy of this in the next step:
Step 4: Connect
In Colab, click the “Connect” button and select “Connect to local runtime”. Enter the URL you just copied and click “Connect”:
That’s it! You now have the Colab research environment running on your local Jupyter server.
Connecting again
Next time you want to connect to a local runtime, you only need to run steps 3 and 4 above.
Security considerations
When you connect to a local Jupyter server, you allow the Colab frontend to execute code in the notebook using local resources, accessing the local file system.
Before attempting to connect to a local runtime, make sure you trust the author of the notebook and ensure you understand the code that is being executed. Don’t run a notebook locally unless you understand the code!
Accessing The Lab
The Lab is available to RW Pro members. Want in? Access to RW Pro is via our Bootcamp program. Join the waitlist here.
Thank you so much Sir