Big Data Engineer

Newron Tech Inc is looking for a Big Data Engineer

Job Posted: 06/17/2024 

Department: IT Software
Project Location(s): Austin, TX

Job Duties:

Analyze and improve stability, efficiency, and scalability of the platform. Manage, build and administer Hadoop clusters using Hortonworks. Create, load and analyze data using Hive queries. Maintain Cluster security with Kerberos, TLS/SSL Encryption, HDFS Encryption. Develop scripts and batch jobs to schedule various Hadoop Program. Actively monitor Hadoop Clusters comprising of many Nodes with Hortonworks distribution with HDP 2.7.4. Perform various configurations, which includes resolving hostnames, user accounts and file permissions, SSH keyless login. Work on Configuring Kerberos Authentication in the cluster. Responsible for cluster maintenance, monitoring, commissioning and decommissioning data nodes, troubleshooting, manage and review data backups. Add and install new components and remove them through Ambari. Create partitions, configure retention policies and replication factors. Manage and review Kafka logs on a daily basis. Import latest version of process groups from Nifi registry in production environment. Involve in updates, version upgrading and patching as part of maintenance activity in different environments. Pull up the data results for the requests from Audit teams. Apply new passwords for Service Ids in many applications across different environments as part of weekly activity. Support technical team members in management and review of Hadoop log Files and data backups. Will work in Austin, TX and/or various unanticipated client sites throughout the U.S. Must be willing to travel and/or relocate.