Nov. 13, 2020, 9:40 a.m.
I have been using CoLab for quite a few years now and have always really appreciated the ability to get access to GPUs (and TPUs) for free. So when I recently found out about CoLab Pro I was reluctant to pay $10 a month for something I had been getting for free. However, at the same time I was paying hundreds of dollars a month for cloud GPU instances. Last week, after going well over my AWS budget last month, I decided to maybe try CoLab Pro and I am very glad I did.
CoLab Pro gives you priority on high-end GPUs - so far I have never not gotten a V100. This is the same GPU I was paying $0.90/hour spot rate (preemptible) on AWS. For me, the main disadvantage of CoLab was that each instance lasted usually about 10 hours before shutting down, and they would time out if left unattended or if I wasn't at the computer. CoLab Pro instances will last up to 24 hours, and they will not time out. I had one running at work the other day and when I got home I figured it had timed out, but when I went back the next morning it was still running !
Obviously, CoLab Pro is better suited to running experiments than executing long training, and it doesn't support multiple GPUs. And if you are using TensorFlow you have TPUs (I prefer PyTorch.) In the past I have repeatedly kicked myself after spending hundreds of dollars training a model, and then finding a small mistake. In the future I will be running my experiments on CoLab Pro and only using VMs when I am sure everything is correct and I need to train models quickly.
Labels: machine_learning , aws , gpu , colab
Oct. 18, 2020, 9:55 a.m.
ModuleNotFoundError: No module named 'mmcv._ext'
If you are getting this error when trying to run the latest version of mmdet with the latest version of mmcv (installed via pip), this solves the problem
pip uninstall mmcv mmcv-full
git clone https://github.com/open-mmlab/mmcv.git
cd mmcv
MMCV_WITH_OPS=1 pip install -e .
Labels: machine_learning , pytorch
Oct. 11, 2020, 11:28 a.m.
So that I can remember next time I need to do it :
lsblk
mkdir /whatever
mount /dev/xvdf /whatever/
Labels: ec2
May 11, 2020, 7:53 a.m.
As of a few days ago they must have changed something in the corporate proxy server at work because all of a sudden no one could push or pull to git remotes. I was getting error "fatal: unable to access [proxy server address]: Timed out"
After much frustration, it turns out there was a fairly simple solution. I had http_proxy and https_proxy environment variables set to the proxy server. If I set an environment variable "no_proxy" with the values of the domains I need to connect to with git everything works fine.
Labels: git