coderplay
11/14/2013 - 7:49 PM
log
Native库用不了
用官方的Hadoop 2.1.0-beta安装后，每次hadoop命令进去都会抛出这样一个Warning
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
设置logger级别，看下具体原因
export HADOOP_ROOT_LOGGER=DEBUG,console
wrong ELFCLASS32，难道是加载的so文件系统版本不对 执行命令file libhadoop.so.1.0.0 libhadoop.so.1.0.0: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped 果然是80386，是32位的系统版本，而我的hadoop环境是64位OS 原来直接从apache镜像中下载的编译好的Hadoop版本native library都是32版本的，如果要支持64位版本，必须自己重新编译，这就有点坑爹了，要知道几乎所有的生产环境都是64位的OS YARN官方对于native library的一段话验证了这一点 “The pre-built 32-bit i386-Linux native hadoop library is available as part of the hadoop distribution and is located in the lib/native directory”
重新checkout source code svn checkout http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta/ 加上编译native的选项，编译时会根据当前的操作系统架构来生产相应的native库 mvn package -Pdist,native -DskipTests -Dtar 再去native文件下查看所有的file type，已经都是64位版的了，替换线上文件，WARNING消除
file *
libhadoop.a: current ar archive
libhadooppipes.a: current ar archive
libhadoop.so: symbolic link to `libhadoop.so.1.0.0'
libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped
libhadooputils.a: current ar archive
libhdfs.a: current ar archive
libhdfs.so: symbolic link to `libhdfs.so.0.0.0'
libhdfs.so.0.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped
Datnode在NameNode同机时启动不了
13/08/29 13:59:38 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
13/08/29 13:59:38 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /usr/local/hadoop/hadoop-2.1.0-beta/lib/native/libhadoop.so.1.0.0: /usr/local/hadoop/hadoop-2.1.0-beta/lib/native/libhadoop.so.1.0.0: wrong ELF class: ELFCLASS32 (Possible cause: architecture word width mismatch)
13/08/29 13:59:38 DEBUG util.NativeCodeLoader: java.library.path=/usr/local/hadoop/hadoop-2.1.0-beta/lib/native/Linux-amd64-64:/usr/local/hadoop/hadoop-2.1.0-beta/lib/native
13/08/29 13:59:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
如果将datanode和namenode布署在同一台机器, 则该机器的datanode会因为DatanodeRegistration(0.0.0.0,...)而被拒绝.
2013-11-14 10:36:18,107 WARN org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Unresolved datanode registration from 172.16.19.32
2013-11-14 10:36:18,108 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mzhou (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-175828921-127.0.0.1-50010-1384408245563, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-ae6d2d3b-d1d2-4de4-8393-520f99933766;nsid=197712146;c=0)
2013-11-14 10:36:18,109 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8020, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.registerDatanode from 172.16.19.32:58126: error: org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-175828921-127.0.0.1-50010-1384408245563, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-ae6d2d3b-d1d2-4de4-8393-520f99933766;nsid=197712146;c=0)
org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-175828921-127.0.0.1-50010-1384408245563, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-ae6d2d3b-d1d2-4de4-8393-520f99933766;nsid=197712146;c=0)
at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:631)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3398)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:881)
at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:90)
at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:18295)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1741)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1737)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1735)
drop table supplier;
drop table lineitem;
drop table part;
drop table partsupp;
drop table customer;
drop table orders;
drop table nation;
drop table region;
create external table supplier (S_SUPPKEY bigint, S_NAME text, S_ADDRESS text, S_NATIONKEY bigint, S_PHONE text, S_ACCTBAL double, S_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/supplier';
create external table lineitem (L_ORDERKEY bigint, L_PARTKEY bigint, L_SUPPKEY bigint, L_LINENUMBER bigint, L_QUANTITY double, L_EXTENDEDPRICE double, L_DISCOUNT double, L_TAX double, L_RETURNFLAG text, L_LINESTATUS text, L_SHIPDATE text, L_COMMITDATE text, L_RECEIPTDATE text, L_SHIPINSTRUCT text, L_SHIPMODE text, L_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/lineitem';
create external table part (P_PARTKEY bigint, P_NAME text, P_MFGR text, P_BRAND text, P_TYPE text, P_SIZE integer, P_CONTAINER text, P_RETAILPRICE double, P_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/part';
create external table partsupp (PS_PARTKEY bigint, PS_SUPPKEY bigint, PS_AVAILQTY int, PS_SUPPLYCOST double, PS_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/partsupp';
create external table customer (C_CUSTKEY bigint, C_NAME text, C_ADDRESS text, C_NATIONKEY bigint, C_PHONE text, C_ACCTBAL double, C_MKTSEGMENT text, C_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/customer';
create external table orders (O_ORDERKEY bigint, O_CUSTKEY bigint, O_ORDERSTATUS text, O_TOTALPRICE double, O_ORDERDATE text, O_ORDERPRIORITY text, O_CLERK text, O_SHIPPRIORITY int, O_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/orders';
create external table nation (N_NATIONKEY bigint, N_NAME text, N_REGIONKEY bigint, N_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/nation';
create external table region (R_REGIONKEY bigint, R_NAME text, R_COMMENT text) using csv with ('csvfile.delimiter'='|') location 'hdfs://172.16.33.40:8020/tpch/tpch100/region';
1 启动CatalogServer 需要配置catalog-site.xml
tajo.catalog.store.class
tajo.catalog.connection.id
public class TajoConstants {
public static String TAJO_VERSION = "0.2.0-SNAPSHOT";
public static String SYSTEM_CONF_FILENAME = "system_conf.xml";
public static String SYSTEM_DIR_NAME = "system";
public static String WAREHOUSE_DIR_NAME = "warehouse";
public static String SYSTEM_RESOURCE_DIR_NAME = "resource";
public static String RESULT_DIR_NAME="RESULT";
private TajoConstants()
}
2013-11-15 23:04:14,440 INFO master.TajoMaster (StringUtils.java:startupShutdownMessage(576)) - STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TajoMaster
STARTUP_MSG: host = wezhang-ld1/127.0.0.1
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.0.5-alpha
STARTUP_MSG: classpath = :/home/mzhou/tajo/latest/conf:/home/mzhou/tajo/latest/conf/*.xml:/home/mzhou/tajo/latest/bin/../tajo-algebra-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-algebra-0.2.0-SNAPSHOT-tests.jar:/home/mzhou/tajo/latest/bin/../tajo-catalog-client-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-catalog-common-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-catalog-server-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-common-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-core-backend-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-core-backend-0.2.0-SNAPSHOT-tests.jar:/home/mzhou/tajo/latest/bin/../tajo-core-pullserver-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-core-storage-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../tajo-core-storage-0.2.0-SNAPSHOT-tests.jar:/home/mzhou/tajo/latest/bin/../tajo-rpc-0.2.0-SNAPSHOT.jar:/home/mzhou/tajo/latest/bin/../lib/activation-1.1.jar:/home/mzhou/tajo/latest/bin/../lib/ant-1.6.5.jar:/home/mzhou/tajo/latest/bin/../lib/antlr-2.7.7.jar:/home/mzhou/tajo/latest/bin/../lib/antlr-3.4.jar:/home/mzhou/tajo/latest/bin/../lib/antlr4-4.1.jar:/home/mzhou/tajo/latest/bin/../lib/antlr4-runtime-4.1.jar:/home/mzhou/tajo/latest/bin/../lib/antlr-runtime-3.5.jar:/home/mzhou/tajo/latest/bin/../lib/aopalliance-1.0.jar:/home/mzhou/tajo/latest/bin/../lib/asm-3.1.jar:/home/mzhou/tajo/latest/bin/../lib/avro-1.5.3.jar:/home/mzhou/tajo/latest/bin/../lib/avro-ipc-1.7.3.jar:/home/mzhou/tajo/latest/bin/../lib/avro-ipc-1.7.3-tests.jar:/home/mzhou/tajo/latest/bin/../lib/avro-mapred-1.7.3.jar:/home/mzhou/tajo/latest/bin/../lib/commons-beanutils-1.7.0.jar:/home/mzhou/tajo/latest/bin/../lib/commons-beanutils-core-1.8.0.jar:/home/mzhou/tajo/latest/bin/../lib/commons-cli-1.2.jar:/home/mzhou/tajo/latest/bin/../lib/commons-codec-1.4.jar:/home/mzhou/tajo/latest/bin/../lib/commons-collections-3.2.1.jar:/home/mzhou/tajo/latest/bin/../lib/commons-configuration-1.6.jar:/home/mzhou/tajo/latest/bin/../lib/commons-daemon-1.0.13.jar:/home/mzhou/tajo/latest/bin/../lib/commons-dbcp-1.4.jar:/home/mzhou/tajo/latest/bin/../lib/commons-digester-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/commons-el-1.0.jar:/home/mzhou/tajo/latest/bin/../lib/commons-httpclient-3.1.jar:/home/mzhou/tajo/latest/bin/../lib/commons-io-2.1.jar:/home/mzhou/tajo/latest/bin/../lib/commons-lang-2.6.jar:/home/mzhou/tajo/latest/bin/../lib/commons-logging-1.1.1.jar:/home/mzhou/tajo/latest/bin/../lib/commons-logging-api-1.1.jar:/home/mzhou/tajo/latest/bin/../lib/commons-math-2.1.jar:/home/mzhou/tajo/latest/bin/../lib/commons-net-3.1.jar:/home/mzhou/tajo/latest/bin/../lib/commons-pool-1.5.4.jar:/home/mzhou/tajo/latest/bin/../lib/core-3.1.1.jar:/home/mzhou/tajo/latest/bin/../lib/datanucleus-connectionpool-2.0.3.jar:/home/mzhou/tajo/latest/bin/../lib/datanucleus-core-2.0.3.jar:/home/mzhou/tajo/latest/bin/../lib/datanucleus-enhancer-2.0.3.jar:/home/mzhou/tajo/latest/bin/../lib/datanucleus-rdbms-2.0.3.jar:/home/mzhou/tajo/latest/bin/../lib/derby-10.8.2.2.jar:/home/mzhou/tajo/latest/bin/../lib/dspace-geoip-1.2.3.jar:/home/mzhou/tajo/latest/bin/../lib/gmbal-api-only-3.0.0-b023.jar:/home/mzhou/tajo/latest/bin/../lib/grizzly-framework-2.1.1.jar:/home/mzhou/tajo/latest/bin/../lib/grizzly-framework-2.1.1-tests.jar:/home/mzhou/tajo/latest/bin/../lib/grizzly-http-2.1.1.jar:/home/mzhou/tajo/latest/bin/../lib/grizzly-http-server-2.1.1.jar:/home/mzhou/tajo/latest/bin/../lib/grizzly-http-servlet-2.1.1.jar:/home/mzhou/tajo/latest/bin/../lib/grizzly-rcm-2.1.1.jar:/home/mzhou/tajo/latest/bin/../lib/gson-2.2.2.jar:/home/mzhou/tajo/latest/bin/../lib/guava-13.0.1.jar:/home/mzhou/tajo/latest/bin/../lib/guice-3.0.jar:/home/mzhou/tajo/latest/bin/../lib/guice-servlet-3.0.jar:/home/mzhou/tajo/latest/bin/../lib/hamcrest-core-1.1.jar:/home/mzhou/tajo/latest/bin/../lib/hcatalog-core-0.5.0-incubating.jar:/home/mzhou/tajo/latest/bin/../lib/hive-exec-0.11.0.jar:/home/mzhou/tajo/latest/bin/../lib/hive-metastore-0.11.0.jar:/home/mzhou/tajo/latest/bin/../lib/hive-serde-0.11.0.jar:/home/mzhou/tajo/latest/bin/../lib/httpclient-4.1.3.jar:/home/mzhou/tajo/latest/bin/../lib/httpcore-4.1.3.jar:/home/mzhou/tajo/latest/bin/../lib/jackson-core-asl-1.8.8.jar:/home/mzhou/tajo/latest/bin/../lib/jackson-jaxrs-1.7.1.jar:/home/mzhou/tajo/latest/bin/../lib/jackson-mapper-asl-1.8.8.jar:/home/mzhou/tajo/latest/bin/../lib/jackson-xc-1.7.1.jar:/home/mzhou/tajo/latest/bin/../lib/jasper-compiler-5.5.23.jar:/home/mzhou/tajo/latest/bin/../lib/jasper-runtime-5.5.23.jar:/home/mzhou/tajo/latest/bin/../lib/JavaEWAH-0.3.2.jar:/home/mzhou/tajo/latest/bin/../lib/javax.inject-1.jar:/home/mzhou/tajo/latest/bin/../lib/javax.servlet-3.0.jar:/home/mzhou/tajo/latest/bin/../lib/javolution-5.5.1.jar:/home/mzhou/tajo/latest/bin/../lib/jaxb-api-2.2.2.jar:/home/mzhou/tajo/latest/bin/../lib/jaxb-impl-2.2.3-1.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-client-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-core-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-grizzly2-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-guice-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-json-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-server-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-test-framework-core-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jersey-test-framework-grizzly2-1.8.jar:/home/mzhou/tajo/latest/bin/../lib/jets3t-0.6.1.jar:/home/mzhou/tajo/latest/bin/../lib/jettison-1.1.jar:/home/mzhou/tajo/latest/bin/../lib/jetty-6.1.14.jar:/home/mzhou/tajo/latest/bin/../lib/jetty-util-6.1.14.jar:/home/mzhou/tajo/latest/bin/../lib/jline-2.11.jar:/home/mzhou/tajo/latest/bin/../lib/jsch-0.1.42.jar:/home/mzhou/tajo/latest/bin/../lib/json-20090211.jar:/home/mzhou/tajo/latest/bin/../lib/jsp-2.1-6.1.14.jar:/home/mzhou/tajo/latest/bin/../lib/jsp-api-2.1-6.1.14.jar:/home/mzhou/tajo/latest/bin/../lib/jsp-api-2.1.jar:/home/mzhou/tajo/latest/bin/../lib/junit-4.10.jar:/home/mzhou/tajo/latest/bin/../lib/kfs-0.3.jar:/home/mzhou/tajo/latest/bin/../lib/libfb303-0.9.0.jar:/home/mzhou/tajo/latest/bin/../lib/libthrift-0.9.0.jar:/home/mzhou/tajo/latest/bin/../lib/log4j-1.2.17.jar:/home/mzhou/tajo/latest/bin/../lib/management-api-3.0.0-b012.jar:/home/mzhou/tajo/latest/bin/../lib/mockito-all-1.8.2.jar:/home/mzhou/tajo/latest/bin/../lib/mockito-core-1.9.5-rc1.jar:/home/mzhou/tajo/latest/bin/../lib/netty-3.6.6.Final.jar:/home/mzhou/tajo/latest/bin/../lib/objenesis-1.0.jar:/home/mzhou/tajo/latest/bin/../lib/org.abego.treelayout.core-1.0.1.jar:/home/mzhou/tajo/latest/bin/../lib/paranamer-2.3.jar:/home/mzhou/tajo/latest/bin/../lib/protobuf-java-2.4.1.jar:/home/mzhou/tajo/latest/bin/../lib/servlet-api-2.5-20081211.jar:/home/mzhou/tajo/latest/bin/../lib/servlet-api-2.5-6.1.14.jar:/home/mzhou/tajo/latest/bin/../lib/servlet-api-2.5.jar:/home/mzhou/tajo/latest/bin/../lib/slf4j-api-1.6.1.jar:/home/mzhou/tajo/latest/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/mzhou/tajo/latest/bin/../lib/snappy-0.2.jar:/home/mzhou/tajo/latest/bin/../lib/snappy-java-1.0.4.1.jar:/home/mzhou/tajo/latest/bin/../lib/ST4-4.0.7.jar:/home/mzhou/tajo/latest/bin/../lib/stax-api-1.0.1.jar:/home/mzhou/tajo/latest/bin/../lib/stringtemplate-3.2.1.jar:/home/mzhou/tajo/latest/bin/../lib/trevni-avro-1.7.3.jar:/home/mzhou/tajo/latest/bin/../lib/trevni-core-1.7.3.jar:/home/mzhou/tajo/latest/bin/../lib/velocity-1.7.jar:/home/mzhou/tajo/latest/bin/../lib/xmlenc-0.52.jar:/home/mzhou/tajo/latest/bin/../lib/zookeeper-3.4.2.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//activation-1.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//asm-3.2.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//avro-1.5.3.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-beanutils-1.7.0.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-beanutils-core-1.8.0.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-cli-1.2.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-codec-1.4.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-collections-3.2.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-configuration-1.6.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-digester-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-el-1.0.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-httpclient-3.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-io-2.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-lang-2.5.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-logging-1.1.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-math-2.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//commons-net-3.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//guava-11.0.2.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//hadoop-annotations-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//hadoop-auth-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jackson-core-asl-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jackson-jaxrs-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jackson-mapper-asl-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jackson-xc-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jasper-compiler-5.5.23.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jasper-runtime-5.5.23.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jaxb-api-2.2.2.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jaxb-impl-2.2.3-1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jersey-core-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jersey-json-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jersey-server-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jets3t-0.6.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jettison-1.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jetty-6.1.26.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jetty-util-6.1.26.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jsch-0.1.42.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jsp-api-2.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//jsr305-1.3.9.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//kfs-0.3.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//log4j-1.2.17.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//paranamer-2.3.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//protobuf-java-2.4.0a.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//servlet-api-2.5.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//slf4j-api-1.6.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//slf4j-log4j12-1.6.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//snappy-java-1.0.3.2.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//stax-api-1.0.1.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//xmlenc-0.52.jar:/home/mzhou/hadoop/latest/share/hadoop/common/lib//zookeeper-3.4.2.jar:/home/mzhou/hadoop/latest/share/hadoop/common//hadoop-common-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/common//hadoop-common-2.0.5-alpha-tests.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/hadoop-hdfs-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/hadoop-hdfs-2.0.5-alpha-tests.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//asm-3.2.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//commons-cli-1.2.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//commons-codec-1.4.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//commons-daemon-1.0.13.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//commons-el-1.0.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//commons-io-2.1.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//commons-lang-2.5.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//commons-logging-1.1.1.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//guava-11.0.2.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jackson-core-asl-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jackson-mapper-asl-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jasper-runtime-5.5.23.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jersey-core-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jersey-server-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jetty-6.1.26.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jetty-util-6.1.26.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jsp-api-2.1.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//jsr305-1.3.9.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//log4j-1.2.17.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//protobuf-java-2.4.0a.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//servlet-api-2.5.jar:/home/mzhou/hadoop/latest/share/hadoop/hdfs/lib//xmlenc-0.52.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//aopalliance-1.0.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//asm-3.2.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//avro-1.5.3.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//commons-io-2.1.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//guice-3.0.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//guice-servlet-3.0.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//hadoop-annotations-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//jackson-core-asl-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//jackson-mapper-asl-1.8.8.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//javax.inject-1.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//jersey-core-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//jersey-guice-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//jersey-server-1.8.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//junit-4.8.2.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//log4j-1.2.17.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//netty-3.5.11.Final.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//paranamer-2.3.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//protobuf-java-2.4.0a.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn/lib//snappy-java-1.0.3.2.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-api-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-applications-distributedshell-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-applications-unmanaged-am-launcher-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-client-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-common-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-server-common-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-server-nodemanager-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-server-resourcemanager-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-server-tests-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-server-web-proxy-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/share/hadoop/yarn//hadoop-yarn-site-2.0.5-alpha.jar:/home/mzhou/hadoop/latest/etc/hadoop:/home/mzhou/hadoop/latest/etc/hadoop/capacity-scheduler.xml:/home/mzhou/hadoop/latest/etc/hadoop/core-site.xml:/home/mzhou/hadoop/latest/etc/hadoop/hadoop-policy.xml:/home/mzhou/hadoop/latest/etc/hadoop/hdfs-site.xml:/home/mzhou/hadoop/latest/etc/hadoop/httpfs-site.xml:/home/mzhou/hadoop/latest/etc/hadoop/yarn-site.xml
STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common -r 1488459; compiled by 'jenkins' on 2013-06-01T04:05Z
STARTUP_MSG: java = 1.6.0_27
************************************************************/
2013-11-15 23:04:14,612 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(278)) - WorkerResourceAllocationThread start
2013-11-15 23:04:14,613 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(209)) - WorkerMonitor start
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mzhou/tajo/tajo-0.2.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mzhou/hadoop/hadoop-2.0.5-alpha/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2013-11-15 23:04:14,663 INFO mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2013-11-15 23:04:14,715 INFO webapp.HttpServer (HttpServer.java:start(333)) - Jetty bound to port
2013-11-15 23:04:14,715 INFO mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - jetty-6.1.14
2013-11-15 23:04:15,044 INFO mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - Started SelectChannelConnector@0.0.0.0:26080
2013-11-15 23:04:15,053 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(201)) - Tajo Root Directory: file:/tmp/tajo-mzhou
2013-11-15 23:04:15,382 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(206)) - FileSystem (file:///) is initialized.
2013-11-15 23:04:15,384 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(226)) - Tajo Warehouse dir: file:/tmp/tajo-mzhou/warehouse
2013-11-15 23:04:15,384 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(235)) - Staging dir: file:/tmp/tajo-mzhou/warehouse
2013-11-15 23:04:15,395 WARN storage.AbstractStorageManager (AbstractStorageManager.java:<init>(85)) - does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled')
2013-11-15 23:04:15,440 INFO service.AbstractService (AbstractService.java:init(81)) - Service:Dispatcher is inited.
2013-11-15 23:04:15,444 INFO catalog.CatalogServer (CatalogServer.java:init(108)) - Catalog Store Class: org.apache.tajo.catalog.store.DerbyStore
2013-11-15 23:04:15,652 INFO store.DerbyStore (AbstractDBStore.java:<init>(67)) - Loaded the Catalog driver (org.apache.derby.jdbc.EmbeddedDriver)
2013-11-15 23:04:15,652 INFO store.DerbyStore (AbstractDBStore.java:<init>(73)) - Trying to connect database (jdbc:derby:/tmp/tajo-catalog-mzhou/db;create=true)
2013-11-15 23:04:16,799 INFO store.DerbyStore (AbstractDBStore.java:<init>(75)) - Connected to database (jdbc:derby:/tmp/tajo-catalog-mzhou/db;create=true)
2013-11-15 23:04:16,955 INFO store.DerbyStore (AbstractDBStore.java:<init>(86)) - The base tables of CatalogServer already is initialized.
2013-11-15 23:04:17,034 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.catalog.CatalogServer is inited.
2013-11-15 23:04:17,034 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.GlobalEngine is inited.
2013-11-15 23:04:17,035 INFO event.AsyncDispatcher (AsyncDispatcher.java:register(153)) - Registering class org.apache.tajo.master.querymaster.QueryJobEvent$Type for class org.apache.tajo.master.querymaster.QueryJobManager$QueryJobManagerEventHandler
2013-11-15 23:04:17,035 INFO service.AbstractService (AbstractService.java:init(81)) - Service:Dispatcher is inited.
2013-11-15 23:04:17,035 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.querymaster.QueryJobManager is inited.
2013-11-15 23:04:17,035 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMasterClientService is inited.
2013-11-15 23:04:17,035 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMasterService is inited.
2013-11-15 23:04:17,035 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMaster is inited.
2013-11-15 23:04:17,035 INFO master.TajoMaster (TajoMaster.java:init(177)) - Tajo Master is initialized.
2013-11-15 23:04:17,035 INFO master.TajoMaster (TajoMaster.java:start(412)) - TajoMaster startup
2013-11-15 23:04:17,035 INFO service.AbstractService (AbstractService.java:start(94)) - Service:Dispatcher is started.
2013-11-15 23:04:17,122 INFO rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (CatalogProtocol) listens on /127.0.0.1:26005
2013-11-15 23:04:17,122 INFO catalog.CatalogServer (CatalogServer.java:start(156)) - Catalog Server startup (127.0.0.1:26005)
2013-11-15 23:04:17,122 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.catalog.CatalogServer is started.
2013-11-15 23:04:17,177 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.GlobalEngine is started.
2013-11-15 23:04:17,177 INFO service.AbstractService (AbstractService.java:start(94)) - Service:Dispatcher is started.
2013-11-15 23:04:17,177 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.querymaster.QueryJobManager is started.
2013-11-15 23:04:17,206 INFO rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (TajoMasterClientProtocol) listens on /127.0.0.1:26002
2013-11-15 23:04:17,206 INFO master.TajoMasterClientService (TajoMasterClientService.java:start(90)) - Instantiated TajoMasterClientService at /127.0.0.1:26002
2013-11-15 23:04:17,206 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMasterClientService is started.
2013-11-15 23:04:17,221 INFO rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (TajoMasterProtocol) listens on /127.0.0.1:26001
2013-11-15 23:04:17,222 INFO master.TajoMasterService (TajoMasterService.java:start(72)) - Instantiated TajoMasterService at /127.0.0.1:26001
2013-11-15 23:04:17,222 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMasterService is started.
2013-11-15 23:04:17,222 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMaster is started.
2013-11-15 23:04:17,276 WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(817)) - fs.default.name is deprecated. Instead, use fs.defaultFStajo> select count(1) from supplier;
Progress: 50%, response time: 1.033 sec
Progress: 100%, response time: 1.481 sec
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mzhou/tajo/tajo-0.2.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mzhou/hadoop/hadoop-2.0.5-alpha/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2013-12-05 20:28:33,330 WARN storage.AbstractStorageManager (AbstractStorageManager.java:<init>(85)) - does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled')
final state: QUERY_SUCCEEDED, init time: 0.135 sec, response time: 1.481 sec
result: hdfs://172.16.19.32:8020/tajo-mzhou/staging/q_1386302792972_0002/RESULT, 1 rows (8 B)
?count
-------------------------------
1000000