Cloudera Solution Architect - Reliable Business technologies Sdn Bhd

Latest Job Circular

Vacancy: 5

Job Context

• Regulatory Alignment: Lead the architectural design to ensure all migrated data structures meet Bank Negara Malaysia (BNM) Project STREAM Phase-1 requirements.
• Workload Rationalization: Conduct a rationalization exercise on the legacy DataStage inventory (approx. 16,000 jobs) to identify and eliminate obsolete logic, aiming for at least a 20% reduction in workload volume.
• Single Source of Truth: Direct the transformation of fragmented data marts into a governed Common Enterprise Data Layer (Silver) to ensure consistent analytical performance across the bank.
• Dual-Stream Governance: Coordinate with the New Source Data Acquisitiion Team to ensure that migrated legacy data and new real-time data integrate seamlessly within the unified Lakehouse.
• Stakeholder Management: Act as the primary technical point of contact for Project Steering Committee and provide detailed technical evidence for regulatory dry runs

Job Responsibilities

• Detailed Technical Requirements
• Core Infrastructure & Modern Lakehouse
• Bare-Metal Architecture: Design and provision CDP Base on bare-metal servers with HDFS and Ozone local storage to eliminate hypervisor overhead and restore data locality.
• Table Format Standardization: Standardize the Lakehouse on Apache Iceberg, implementing ACID-style consistency, row-level updates/deletes, and schema evolution.
• Serving Layer Optimization: Configure Impala + Kudu for high-concurrency, subsecond OLAP analytics to replace and outperform legacy Netezza BI workloads.
• Workload Modernization & Automation
• ETL Re-Engineering: Lead the automated translation of DataStage job logic into Spark SQL and DataFrame operations using LLM-based code accelerators.
• Complex Refactoring: Manually refactor complex Netezza Stored Procedures and User Defined Functions (UDFs) that cannot be handled by automated tools.
• Orchestration Modernization: Migrate legacy ZENA workflows and shell scripts to Apache Airflow DAGs, implementing built-in retries, backfills, and SLA monitoring. C. Governance, Security, & Quality
• Access Control: Configure Apache Ranger and Ranger KMS for fine-grained security (RBAC/ABAC) and encryption key management.
• Data Quality Gates: Embed Great Expectations (GX) validation gates within Airflow and NiFi pipelines to automate source-to-target reconciliation (checksums, row counts, and hash totals)

Employment Status

Full time

Educational Requirements

• Bachelor in Engineering (BEngg) in in Data Science, Applied Statistics and Data Science, Information Technology
• Skills Required:

Experience Requirements

• Minimum 6 year(s)

Additional Requirements

• Age 24 to 45 years

Workplace

• Work from office

Job Location

Salary

• TK. 250000 - 350000 (Monthly )

Compensation & other benefits

• Lunch Facilities: Full Subsidize
• Festival Bonus:
• Salary Review: Yearly

Job Source

- MoreJobs.com Online Job Posting.