尋找Google的創新

Docker專案在GCE上也積極的在發展，可以參考 https://docs.docker.com/installation/google/ 透過Google Cloud啟動Docker的Playground Server，可以快速的體驗Docker的功能喲～ 首先看看Docker Project提供的Image $ gcutil --project=google-containers listimages +--------------------------------------------------------------------------+-------------+--------+ | name                                                                     | deprecation | status | +--------------------------------------------------------------------------+-------------+--------+ | container-vm-v20140826                 ...

下面是簡單的Cloud Storage切割上傳＋重組的過程 給大家參考： 1. Split a huge file use "split -b [size] [file]" 2. upload to cloud storage us parameter "-m" 3. check the process forked, there will be more than one upload processes 4. Compose the uploaded object in cloud storage 最後，如果希望把不必要的檔案砍掉 可以用： gsutil rm -rf gs://[your-bucket]/x*

今天突然發現Compute Engine的新功能... 居然可以透過頁面提供SSH Console!!! (雖然做法不難，不過Google真是有誠意拉！) 直接在machine detail頁面，展開最上方SSH右邊的選單，選擇"Open in browser window" 即可開啟SSH視窗 SSH動作是透過cloudssh這檯主機來做SSH轉導服務，當然，事前需要透過該Server進行SSH key的產生... Key產生後，會建立一筆臨時使用的SSH金鑰，然後就可以直接連線.... 金鑰的一樣會塞到~/.ssh/authorized_keys，格式如下： 仔細看，public key後面會有： google-ssh {"userName":" simonsu@mitac.com.tw ","expireOn":"2014-07-15T14:13:40+0000"} 有寫上expire time...，看起來僅作為一次登入使用... 測試一下最麻飯的vi模式，也通用喔！ 真是要稱讚一下Google，厲害啊！

Google Cloud Storage終於有神人提供fuse工具拉，也就是說可以透過mount的方式來掛載google coud storage (GCS)成為filessystem，下面是測試的狀況 Download & Install  下載點： gcsfs-downloads (on Google Drive) . In Local: $ gcutil push [your-instance-name] ~/Downloads/gcsfs-0.15-1.x86_64.rpm [your-instance-folder] $ gcutil ssh [your-instance-name] In remote: $ cd [your-instance-folder] $ sudo yum localinstall gcsfs-0.15-1.x86_64.rpm Initial Generate Access Token $ mkdir .gcsfs $ gcsfs_gs_get_token ~/.gcsfs/gs.token Paste this URL into your browser: https://accounts.google.com/o/oauth2/auth?client_id=591551582755.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&scope=https%3a%2f%2fwww.googleapis.com%2fauth%2fdevstorage.read_write&response_type=code Please enter the authorization code: 4/4VNfqTct.....mbpQjgI <--在這邊填入授權碼 curl_easy_handle::pre_init: ssl version: NSS/3.15.3 Done! Check generated token $ cat .gcsfs/gs....

還在使用NFS嗎？似乎有更好的解決方案了噢！ http://thegeekinthecorner.wordpress.com/2013/11/24/gcsfs-a-fuse-driver-for-google-cloud-storage-now-available/ 趕快來試試看！

在stackoverflow上回報後，很快地收到了Mike的回覆： https://stackoverflow.com/questions/23368191/compute-engine-use-gsutil-to-download-tgz-file-has-crcmod-error The object you're trying to download is a composite object ( https://developers.google.com/storage/docs/composite-objects ), which basically means it was uploaded in parallel chunks. gsutil automatically does this when uploading objects larger than 150M (a configurable threshold), to provide better performance. Composite objects only have a crc32c checksum (no MD5), so in order to validate data integrity when downloading composite objects, gsutil needs to perform a crc32c checksum. Unfortunately, the libraries distributed with Python don't include a compiled crc32c implementation, so unless you install a compiled crc32c, gsutil will use a non-compiled Python implementation of crc32c that's quite slow. That warning is printed to let you know there's a way to fix that performance problem: Please run: gsutil help crcmod ...

關於GCE上面主機的crcmod錯誤... 目前尚不知道原因為何...(目前觀察都發生在有壓縮檔案的部分) 不過在gsutil工具上可以先透過/etc/boto.cfg上面加上check_hashes = never來略過crcmod的檢查... 錯誤發生： $ gsutil cp gs://mybucket/data.tgz . Copying gs://mybucket/data.tgz... CommandException: Downloading this composite object requires integrity checking with CRC32c, but your crcmod installation isn't using the module's C extension , so the the hash computation will likely throttle download performance. For help installing the extension, please see:   $ gsutil help crcmod To download regardless of crcmod performance or to skip slow integrity checks, see the " check_hashes " option in your boto config file. 修改boto設定 $ vi /etc/boto.cfg [GSUtil] default_project_id = 429100748693 default_api_version = 2 check_hashes = never ...