cztchoice
7/15/2015 - 2:58 AM

get content in web in various language

get content in web in various language

Because of some reason (Maybe the debug memory of web? I cannot recall it. I think there are a lot things I need to know to use it well, so I just don't use it.), I think get content from web is very very hard. So I create this compilation to make me realize that get web conten just another sentence, not monster.

I always mix the http get and cache, think it is one operation, Because it can do so many things, I think it must be very difficult, which make me hard to start to use it.

Python using urllib2:

url=r'http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest'
data=urllib2.urlopen(url).read()

C/C++ using libcurl: Code from here: http://stackoverflow.com/questions/389069/programmatically-reading-a-web-page#389074

 #include <stdio.h>
 #include <curl/curl.h>

 int main(void)
 {
   CURL *curl;
   CURLcode res;

   curl = curl_easy_init();
   if(curl) {
     curl_easy_setopt(curl, CURLOPT_URL, "curl.haxx.se");
     res = curl_easy_perform(curl);
      /* always cleanup */
    curl_easy_cleanup(curl);
   }
   return 0;
 }

Go:

package main

import (
    "io/ioutil"
    "net/http"
    "fmt"
)

func main() {
  resp, err := http.Get("http://book.douban.com/subject/26416401/")
  if err != nil {
    panic(err)
  }

  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
  // resp.Body.Close()
  if err != nil {
    panic(err)
  }
  fmt.Printf("%s", body)
}

Go without error handling:

package main

import (
    "io/ioutil"
    "net/http"
    "fmt"
)

func main(){
  resp, _ := http.Get("http://book.douban.com/subject/26416401/")
  defer resp.Body.Close()
  body, _ := ioutil.ReadAll(resp.Body)
  fmt.Printf("%s", body)
}

lua:

You need run luarocks install socket before

local http=require'socket.http'
body,c,l,h = http.request('http://book.douban.com/subject/26416401/')
print('status line',l)
print('body',body)

bash:

curl http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest
or
wget -q -O- http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest