Recently, I have rent a server on Aliyun to set up websites and play around with web service, and it’s very exciting to have a virtual host to show some awesome stuff. However, due to some government regulations, you cannot visit some websites, invoke some web service abroad or even install some python packages if you’re in China. This post details the process of setting up a HTTP proxy in the terminal and speed up git, pip, curl, etc.

Step 1: Install Shadowsocks Client

Shadowsocks is an open-source proxy project to help people visit some websites :). To speed up web surfing, we need to install a Shadowsocks client first.

For Ubuntu:

1
sudo apt-get install shadowsocks

For Mac:

1
brew install shadowsocks-libev

You can also download a GUI client from https://sourceforge.net/projects/shadowsocksgui/

Create a directory named shadowsocks: mkdir shadowsocks, cd into it, create a file named shadowsocks.json with the following content:

1
2
3
4
5
6
7
8
{
"server": "139.162.10.88", // your server ip address
"server_port": 12345, // your server port
"local_port": 1080,
"password": "your_password",
"timeout": 600,
"method": "aes-256-cfb" // your encrption method
}

Run the following command to start the Shadowshocks client:

1
sudo nohup sslocal -c ./shadowsocks.json &

It’ll open a local port to wait for the connection, and here it’s 1080.

Step 2: Use Shadowsocks Client as HTTP Proxy

First, let’s do some check:

1
2
$ curl ip.gs
当前 IP:59.110.66.150 来自:中国北京北京 阿里云/电信/联通/移动/铁通/教育网

Yes, I’m safely protected by the wall :).

With the environment variable http_proxy set, we can travel abroad:

1
2
$ http_proxy=socks5://localhost:1080 curl ip.gs
当前 IP:139.162.10.88 来自:新加坡新加坡 linode.com

However, some command line tools (such as npm) don’t support socks5 protocol, and under the help of polipo we can convert socks5 into http.

For Ubuntu:

1
sudo apt-get install polipo

Edit the config file at /etc/polipo/config, and append the following two lines:

1
2
socksParentProxy = "localhost:1080"
socksProxyType = socks5

Finally, restart polipo:

1
2
sudo service polipo stop
sudo service polipo start

And for Mac:

1
brew install polipo

Edit file /usr/local/opt/polipo/homebrew.mxcl.polipo.plist, add the socksParentProxy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>homebrew.mxcl.polipo</string>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>ProgramArguments</key>
<array>
<string>/usr/local/opt/polipo/bin/polipo</string>
<string>socksParentProxy=localhost:1080</string>
</array>
<!-- Set `ulimit -n 65536`. The default macOS limit is 256, that's
not enough for Polipo (displays 'too many files open' errors).
It seems like you have no reason to lower this limit
(and unlikely will want to raise it). -->
<key>SoftResourceLimits</key>
<dict>
<key>NumberOfFiles</key>
<integer>65536</integer>
</dict>
</dict>
</plist>

Restart polipo:

1
2
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.polipo.plist
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.polipo.plis

Also you can make it start when the system launches:

1
2
3
ln -sfv /usr/local/opt/polipo/*.plist ~/Library/LaunchAgents

Now you can the the http proxy offered by polipo:

1
2
$ http_proxy=http://localhost:8123 curl ip.gs
当前 IP:139.162.10.88 来自:新加坡新加坡 linode.com

You can set it globally by (here I just add https_proxy by the way):

1
2
export http_proxy=http://localhost:8123
export https_proxy=http://localhost:8123

And stop the proxy with unset http_proxy or (unset https_proxy).

Also, you can put the two export lines into your .zshrc(or, .bashrc) to set the proxies automatically when you start a new shell session.

source ~/.zshrc or restart a new session, you can see the proxy works:

1
2
$ curl ip.gs
当前 IP:139.162.10.98 来自:新加坡新加坡 linode.com

Step 3: Set Up git Proxy

First try it out:

1
git clone https://github.com/twbs/bootstrap.git

In Aliyun, it’s quite slow. We can let git use the proxy with the following command:

1
git config --global http.proxy 'socks5://127.0.0.1:1080'

It will create a http section in your ~/.gitconfig file:

1
2
[http]
proxy = socks5://127.0.0.1:1080

Now, you can clone any repository in Github much faster.

Step 4: Set Up pip Proxy

In Aliyun, the connection of pip is very slow:

1
2
3
4
$ pip install tornado
Collecting tornado
Downloading tornado-4.4.2.tar.gz (460kB)
4% |█▍ | 20kB 11kB/s eta 0:00:37

If you set the https_proxy environment variable, the following steps are saved. However, since proxychains is very handy, I’ll spend one section to detail it’s configuration.

For Ubuntu:

1
sudo apt-get install proxychains

Change the config file at ~/.proxychains/proxychains.conf, and change the [ProxyList] in it:

1
2
3
4
[ProxyList]
socks5 127.0.0.1 1080
# You can also use the http proxy polipo offers
# http 127.0.0.1 8123

For Mac, use proxychains-ng instead:

1
$ brew install proxychains-ng

Change the config file at /usr/local/etc/proxychains.conf, and change the [ProxyList] in it like what we did for Ubuntu.

Now you can install packages much faster.

For Ubuntu, put proxychains before pip:

1
2
3
4
5
6
$ proxychains pip install tornado
ProxyChains-3.1 (http://proxychains.sf.net)
Collecting tornado
|D-chain|-<>-127.0.0.1:1080-<><>-151.101.36.223:443-<><>-OK
Downloading tornado-4.4.2.tar.gz (460kB)
51% |████████████████▍ | 235kB 553kB/s eta 0:00:01

For Mac, use proxychains4 instead:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
proxychains4 pip install Faker
[proxychains] config file found: /usr/local/Cellar/proxychains-ng/4.11/etc/proxychains.conf
[proxychains] preloading /usr/local/Cellar/proxychains-ng/4.11/lib/libproxychains4.dylib
[proxychains] DLL init: proxychains-ng 4.11
[proxychains] DLL init: proxychains-ng 4.11
Collecting Faker
[proxychains] Strict chain ... 127.0.0.1:1080 ... pypi.python.org:443 ... OK
Downloading Faker-0.7.7-py2.py3-none-any.whl (562kB)
100% |████████████████████████████████| 563kB 1.3MB/s
...
Installing collected packages: Faker
Successfully installed Faker-0.7.7

Step 5: Set npm Proxy

If you set the shell http_proxy and https_proxy, npm would take advantage of the proxy. But you can always make things safer by instruct the following commands:

1
2
npm config set proxy http://localhost:8123
npm config set https-proxy http://localhost:8123

Also with the help of proxychains, you can make npm traffic go through the proxy. But you should make sure not use proxychains with the http_proxy and https_proxy environmant variables set at the same time.

Conclusion

We can live a safe and slow life, or a “dangerous” but fast one, and Shadowsocks with all these proxy tools will help us to achieve that. Thanks all these “wanderlusters” who wallow in freedom to create these awesome stuff, and wish you a good journey with the “ladder”. Beyond the wall, there is the world.