
Big Data Platform Engineering Specialist
City : Waterloo
Category : Software & Systems
Industry : IT
Employer : BlackBerry
Worker Sub-Type:
RegularJob Description:
POSITION SUMMARY AND ACCOUNTABILITIES
The successful candidate, as part of the Big Data Infrastructure Platform Engineering team, will be working closely with a team of talented architects and engineers to deliver scalable, highly available, and cost-efficient Big Data platform solutions integrated in the corporate and BlackBerry services design in both the lab and in production. The following are the day-to-day responsibilities of the role:
- Drive new solutions and related architectural initiatives to enrich the Big Data platform service catalogue within AWS.
- Develop and test deployment architectures, with varying scale, and document them for consistent use and efficient configuration management.
- Conduct failure mode analysis, develop and test recovery procedures, and invoke failover and DR plans when required.
- Deploy, upgrade and patch the Big Data systems in production and verify the platform health upon changes.
- Define and maintain the Big Data platform key performance indices, system health thresholds, and known issues list to enable the NOC for monitoring and incident management.
- Troubleshoot Hadoop/Kafka problems, drive corrective actions, and platform improvement plans.
- Automate deployment procedures and platform templates to increase maintenance and operational efficiency.
- Work closely with the services team and the Product development team to tune and scale the Big Data platform according to growth trends and projection plans.
- Optimize the Big Data platform performance and system resources allocation to satisfy the application requirements and improve the cost model.
The minimum requirement for this position is 2 years of technical experience and support of AWS EMR and a Bachelor degree in Computer Science/Engineering or equivalent.
TECHNICAL REQUIREMENTS
- Kafka
- AWS EMR
- AWS MSK
- Yarn
- Oozie
- ZooKeeper
- Ambari Views
- Hive
- Solr
- Apache Ranger
ESSENTIAL SKILLS AND QUALIFICATIONS
- Certifications required: AWS Solutions Architect or AWS SysOps Administrator
- Proven experience installing, tuning and patching AWS EMR services
- Experience in designing and configuring enterprise-class scalable fault-tolerant architectures
- Strong systems administration, preferably Ubuntu, CentOS, and Windows.
- Knowledge of AWS cloud services
- Reasonable scripting skills in automating procedures (e.g. shell, SED, AWK, Perl, Python, etc.)
- Demonstrated ability to proactively build and manage relationships
- Must be able to work well within a team environment (local and remote members), and interface well with other teams
- Strong written and verbal communication skills
ADDITIONAL ASSETS
- Familiarity with Java support/test tools and resources, Java Message Queuing
- Experience with No-SQL databases like Cassandra, Hbase, ElasticSearch, etc
- Familiarity with software development processes and tools
- LDAP administration
- Web server administration (Apache, tomcat)
#LI-NR1
Scheduled Weekly Hours:
40