SUMMARY

• 7年以上IT工作经验，4年以上Hadoop生态系统实施经验, maintenance, ETL and Big Data analysis operations.
• Excellent understanding / knowledge of BigData and Hadoop ecosystems.
• Experience with configuration of Hadoop ecosystem components such as MapReduce, HDFS, HIVE, HBase, PIG, Sqoop, Zookeeper, Oozie and Flume.
• Experienced in using NFS (network file systems) for Name node metadata backup.
• In depth knowledge of Hadoop architecture and various components such as HDFS, JobTracker, NameNode, DataNode and MapReduce concepts.
• Good Exposure to Map Reduce programming, PIG Scripting and Distributed Application and NoSQL Databases like HBase, Mongo DB.
• Experience in maintaining multiple Hadoop clusters (prod, dev etc.).
• 有收集和分析现有基础设施的Hadoop集群性能调优经验.
• Experience in automating the Hadoop installation, configuration and maintaining the cluster by using the tools like puppet.
• Experience in managing and monitoring Hadoop cluster using Cloudera Manager.
• 了解使用Oozie设计时间驱动和数据驱动的自动化工作流程.
• Excellent analytical, multi - tasking, problem-solving, time-management & communication skills with particular emphasis on clearly communicating & documenting detailed Test Requirements & Tests.
• Experience in support analysts by administering and configuring Hive.
• 有支持数据分析师运行Pig和Hive查询的经验.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Developed MapReduce programs to perform analysis.
• Performed importing and exporting data into HDFS and Hive using Sqoop.
• 有编写shell脚本的经验，将MySQL服务器上的共享数据转储到HDFS.
• 具有核心Java, Java虚拟机(JVM)和多线程处理经验.
• 作为Netcool管理服务(NMS)顾问超过3年的丰富经验, 参与为客户开发故障管理领域的体系结构解决方案, Performance Management, 运营支持系统(OSS)和其他相关学科包括IBM Netcool.
• 负责使用ambari监控hadoop集群，为故障排除配置各种警报, setting parameters for enhancing performance.
• 具有基于Web的门户系统的使用经验，例如使用Edge工具(JARVIS)，该工具可以显示并将启用Web的网络管理应用程序整合到单个视图中.
• Experience in Unix Shell Scripting, SQL, Reporting and validating complex Stored Procedures, Triggers.
• 在UNIX平台上工作，对VI编辑器和UNIX命令有了清晰的了解.
• 优秀的沟通和人际交往能力，自我激励，注重细节.

TECHNICAL SKILLS

Operating Systems: Linux, Windows XP, Server 2003, Server 2008

Databases: Oracle, MySQL, MS SQL Server 2000, DB2

Hadoop Ecosystem Development: HDFS, HBase, MapReduce, YARN, Hive, Pig, Sqoop, Flume, Oozie, Zookeeper

Fault Management Tool: IBM Tivoli suite of products (OMNIbus, ITNM, Web-GUI, TBSM, TCR, Impact, ITM

Languages: C, Java, SQL, Pig Latin, Unix Shell Scripting

IDE/ Tools: Eclipse

Others: JARVIS, HP-BSM

PROFESSIONAL EXPERIENCE

Confidential, PA

Hadoop Developer

Responsibilities:

• Analyzing the requirement to setup a cluster.
• Installed and configured Hadoop, MapReduce, HDFS (Hadoop Distributed File System), developed multiple MapReduce jobs in java.
• Worked with the infrastructure and admin team in designing, modelling, sizing and configuring Hadoop cluster of 15 nodes.
• 在Java中开发Map Reduce程序，用于解析原始数据和填充staging table.
• 创建Hive查询，将原始数据与EDW参考表进行比较，并执行聚合
• Importing and exporting data into HDFS and Hive using Sqoop.
• Experienced in analyzing data with Hive and Pig.
• Writing Pig scripts to process the data.
• 开发PIG拉丁脚本，从web服务器输出文件中提取数据，加载到HDFS.
• Integrating bulk data into Cassandra file system using MapReduce programs.
• Got good experience with NOSQL database.
• 参与HBASE的设置，并将数据存储到HBASE中，用于后续的分析.
• Experienced in managing and reviewing Hadoop log files.
• Experienced in defining job flows.
• Experienced in managing and reviewing Hadoop log files.
• Installed and configured Hive and also written Hive UDFs.
• Involved in creating Hive tables, 使用HiveQL加载数据和编写hive查询，HiveQL将以map reduce的方式在内部运行.
• Extracted the data from MySQL into HDFS using Sqoop.
• 使用HiveQL分析分区和桶数据，并计算用于报告的各种指标.
• Deployed Hadoop Cluster in Fully Distributed and Pseudo-distributed modes.
• Experience in managing and monitoring Hadoop cluster using Cloudera Manager.
• 支持用Pig设置QA环境和更新实现脚本的配置, Hive and Sqoop.
• 单元测试了一个原始数据样本，改进了性能，并移交给生产.

Environment: Java (JDK1.7), Java 7, Hadoop 2.6.0, MapReduce, HDFS, Hive 0.13.0, Sqoop 1.4.4, HBase, Pig 0.12.0、Oozie、Kerberos、Linux、Shell Scripting、Oracle 11g、PL/SQL、SQL*PLUS、HDInsight

Confidential

Hadoop Developer

Responsibilities:

• Installed and configured Hadoop MapReduce, HDFS, 用Java开发了多个MapReduce作业，用于数据清理和预处理.
• 安装和配置Apache Hadoop集群，用于应用程序开发和Hive等Hadoop工具, Pig, HBase and Sqoop.
• 对Hadoop栈，Hive, Pig和Map/Reduce有很好的理解和相关经验.
• Wrote MapReduce jobs to discover trends in data usage by users.
• Involved in defining job flows.
• Involved in database connection by using SQOOP
• 使用HiveQL创建Hive表，加载数据，编写Hive查询.
• Experienced in SQL DB Migration to HDFS
• Got good experience with NOSQL database.
• Used Oozie to orchestrate the workflow.
• Installed and configured Pig for ETL jobs.
• 使用大数据技术，并有Hadoop分布式文件系统的实践经验.
• Worked on Map Reduce programming and Pig latin scripts.
• 有使用Apache Spark使用Core分布式计算和数据挖掘库的经验.
• Integrating bulk data into Cassandra file system using MapReduce programs.
• 参与使用Cassandra Query Language为客户数据创建数据模型.
• The data is collected from distributed sources into Avro models. 应用转换和标准化，并加载到HBase中进行进一步的数据处理.

Environment: Hadoop, Hive, Linux, MapReduce, HDFS, Hive, Python, Pig, Sqoop, Cloudera, Cassandra, Spark, Shell Scripting, Java (JDK 1.6), Java 6, Oracle 10g, PL/SQL, SQL*PLUS

Confidential, Oklahoma, OK

Hadoop Developer/ Administrator

Responsibilities:

• 从业务伙伴和主题专家那里收集业务需求.
• Involved in installing Hadoop ecosystem components.
• Responsible to manage data coming from different sources.
• Monitoring the jobs to analyze performance statistics.
• Performing Unit Testing of completed jobs.
• Involved in database connection by using SQOOP.
• Applying optimization techniques at both Hadoop and Database level.
• 参与创建Hive表，以及使用Hive查询加载和分析数据.
• Developed Simple to complex MapReduce Jobs using Hive and Pig.
• 参与运行Hadoop作业，处理数百万条文本数据记录.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required .
• Experienced in defining job flows.
• Experienced in managing and reviewing Hadoop log files.
• Extracted files from MongoDB through Sqoop and placed in HDFS and processed.
• 有运行Hadoop流作业处理tb级XML格式数据的经验.
• 加载和转换大型结构化、半结构化和非结构化数据集.
• Automation script to monitor HDFS and HBase through cronjobs.
• Plan, design, and implement processing massive amounts of marketing information, complete with information enrichment, text analytics, and natural language processing.
• 准备多集群测试工具来测试系统的性能和故障转移.

Environment: HadoopHDFS、Map Reduce、Hive、Pig、Sqoop、Oozie、HBase、Linux、Java、Xml、MongoDB.

Confidential, CA

Java Developer

Responsibilities:

• 积极参与各个模块的分析、设计、开发和测试阶段.
• 在整个项目过程中，每天与团队成员一起认真参与敏捷站立会议，使用Rally软件更新项目状态.
• Estimated Level1 and Level2 and Designed Class, Sequence Diagrams.
• Worked on J2EEdesign patternslikeDAO, Front Controller, Session Façade, Factory Pattern and Singleton.
• Created mobile web applications using HTML5, CSS3, JSON, and JavaScript.
• Worked on OOPS concepts, Collections Framework using Core Java.
• Extensively used Spring MVC architecture for the application development.
• 使用Spring IOC开发应用程序服务组件和配置bean .
• 通过使用消息队列中的消息，使用JMS进行异步通信.
• 使用Hibernate作为数据持久化层，使用Hibernate映射文件配置服务和持久化对象.
• Involved in frameworks integration of Spring and Hibernate.
• 生产和消费基于REST的web服务，使用SOAPUI工具进行单元测试并生成json.
• Worked on Database queries using Oracle instance.
• Used JAXB to convert XML content into a Java Object.
• Involved in unit testing, 根据适用的开发标准进行测试用例评审和代码评审.
• Involved in integration system testing and user acceptance testing (UAT)
• Worked on Web deployments in DEV, QA and UAT environments.
• Support the Application whenever encountered Production issues.
• Deployed application in Weblogic Application server.
• SVN is used as Code Versioned control.
• Firebug is used as debugger.
• Used Maven script to build application.

Environment: Core Java1.6, Oracle10g, Toad, JSP, Spring3.0, Hibernate4.0, JavaScript, HTML5, jQuery, Ajax, Apache CXF Framework, REST Webservices, JAXB, Linux, Shell scripting, WinSCP, log4J, Maven, SVN, Weblogic Server11g, Eclipse, Microsoft Visio.

Confidential

Java Developer

Responsibilities:

• Participates in the fullSDLC including collaborative review meetings, estimations, design sessions, development and testing.
• Developed the User Interface using JSP/AJAX/ HTML / CSS/ Java Script.
• Designed and developedUse Cases, Class diagramsandSequence Diagram.
• 实现了多种j2ee设计模式，如dao、DTO、工厂模式和单例模式.
• Developed the application using Java Server Faces (JSF) and Spring MVC.
• Implement GET and POST operations using Restful web services.
• 完全参与使用DOM和JaxB构建组件，这有助于解析XML.
• Written JDBC programs for callable statements, 编写用Oracle编写的语句，并使用TOAD工具与数据库交互.
• UsedCVSfor version controlling.
• Used log4j for logging statements.

Environment: Core Java 1.5, JSF1.1, Spring 2.0, Restful webservices, Rest Client, XML, XSD, JAXB, DOM, Tomcat, JSP, Oracle 10g, SQL/PL SQL, Hibernate2.0, Java script, CSS, HTML, Ajax, log4j, My Eclipse, Toad, Ant1.7, Microsoft Visio.