Software Engineer II - Java/Python, UI, AWS, LLM

📍 Job Overview

Job Title: Software Engineer II - Java/Python, UI, AWS, LLM Company: JPMorgan Chase & Co. Location: Bengaluru, Karnataka, India Job Type: Full time Category: Software Engineering / Technology Date Posted: 2026-02-11T06:46:21 Experience Level: 2-5 Years (Mid-level) Remote Status: On-site

🚀 Role Summary

• This role focuses on the design, development, and troubleshooting of cutting-edge LLM-powered applications and services within a regulated financial environment.
• Key responsibilities include implementing secure, high-quality production code for microservices and inference pipelines, with a strong emphasis on LLM operations (LLMOps) best practices.
• The position involves leveraging LLMs for enhanced software solutions, ensuring data quality, implementing guardrails, and providing Level 3 support for production systems.
• Candidates will work in an agile team environment, contributing to the enhancement and delivery of state-of-the-art technology products for the Commercial & Investment Bank’s Markets Tech Team.

📝 Enhancement Note: This role is specifically for a Software Engineer II, indicating a mid-level position requiring practical experience beyond entry-level. The emphasis on LLM, AWS, Java/Python, and regulated environments suggests a specialized track within software engineering focused on AI integration in financial services. The "Markets Tech Team" context implies a focus on trading, investment, or financial market technology.

📈 Primary Responsibilities

• Execute creative LLM-assisted software solutions, focusing on the design, development, and troubleshooting of LLM-powered applications and services, including retrieval-augmented generation (RAG) and agent workflows.
• Develop and enforce data quality rules, guardrails for prompts, and post-processing mechanisms, such as PII redaction, toxicity filters, hallucination mitigation, and output schema validation, to ensure regulatory compliance and model safety.
• Provide Level 3 (L3) support for LLM-assisted production systems, owning complex incidents, managing model and prompt rollouts/rollbacks, and addressing dependency issues to ensure high availability, reliability, and adherence to SLAs, including latency and cost budgets.
• Support Business As Usual (BAU) operations for Markets businesses by maintaining and evolving LLM use cases that support market workflows, utilizing disciplined change management, canary releases, A/B tests, and close collaboration with product, controls, and operations teams.
• Create secure, high-quality production code by implementing LLM-assisted microservices, synchronous and asynchronous inference pipelines, deterministic fallbacks, circuit breakers, and robust observability for production reliability.
• Produce architecture and design artifacts, including model cards, system/data lineage, RAG/agent reference architectures, prompt libraries, versioning strategies, and evaluation plans, ensuring design constraints and regulatory expectations are met.
• Identify and analyze hidden problems and patterns using telemetry, error analysis, prompt and context analytics, and drift detection to optimize model selection, prompt strategies, retrieval quality, chunking/embedding strategies, and system architecture.
• Drive LLM Ops best practices by integrating models, prompts, and evaluations into CI/CD pipelines, enforcing approvals, segregation of duties, and reproducibility, and automating regression and guardrail tests across environments.
• Ensure a deep understanding of the strengths, limitations, and risk profiles of approved LLMs (e.g., Claude, ChatGPT) and design multi-agent workflows incorporating LLM-driven analysis, code generation, testing, and review with explicit human approval gates and segregation of duties.
• Ensure LLM-driven systems meet enterprise reliability and resilience expectations, including disaster recovery, fallback behaviors, regional resiliency, and performance Service Level Objectives (SLOs).

📝 Enhancement Note: The responsibilities heavily emphasize LLM integration, LLMOps, and adherence to strict financial industry regulations. This goes beyond typical software engineering by requiring specialized knowledge in AI model management, prompt engineering, and ensuring system robustness in a high-stakes environment. The focus on "Agentic AI" and "novel Agentic AI way" indicates a forward-looking approach to AI development.

🎓 Skills & Qualifications

Education:

• Formal training or certification in software engineering concepts is required.
• A Bachelor's degree in Computer Science, Engineering, or a related field is typically expected for a Software Engineer II role, though not explicitly stated, it can be inferred.

Experience:

• Minimum of 2 years of applied experience in software engineering.
• Practical experience applying software engineering concepts to LLM-enabled systems in regulated environments is essential.
• Experience with system design, application development, and ensuring operational stability for LLM architectures is required.

Required Skills:

• Strong coding proficiency in Java and Python, applied to building LLM-enabled microservices, retrieval pipelines, evaluators, and data tooling.
• Solid understanding of data structures, algorithms, and object-oriented programming, specifically as they relate to LLM latency, caching, and throughput optimization.
• Hands-on experience with AWS and cloud data management services (e.g., Redshift, DynamoDB, Aurora, Databricks).
• Experience integrating managed model endpoints and embedding/vector services.
• Familiarity with secure secret management, networking principles, and least-privilege access controls.
• Proficiency in automation, CI/CD, and agile methodologies, with extensions for LLMOps (e.g., prompt and config versioning, automated evaluations, canary releases, rollback strategies).
• Strong analytical and problem-solving skills, with the ability to identify and resolve complex technical issues.
• Excellent communication skills, including the ability to explain model behaviors, tradeoffs, and control decisions to both technical and non-technical stakeholders.
• Experience in providing Level 3 (L3) and BAU support for markets businesses, leveraging LLMs for incident triage, run book retrieval, and pre-approved auto-remediation.
• Expert-level knowledge of how Large Language Models (LLMs) work, including hands-on experience training and fine-tuning approved models (e.g., Claude, ChatGPT, and successors).
• Proven track record integrating LLMs as controlled, reliable components within the software engineering lifecycle in regulated environments, ensuring determinism, reproducibility, safety, and traceability.
• Strong understanding of data modeling challenges in big data and LLM contexts, including embeddings, chunking strategies, vector similarity nuances, retrieval quality measures, and document lineage.

Preferred Skills:

• Experience defining model usage guidelines for requirements analysis, code generation/refactoring, test generation, and documentation.
• Ability to lead the use of LLMs for structured requirements analysis, translating business and regulatory requirements into clear technical specifications and control implementations.
• Experience establishing best practices for prompt-driven design and development, treating prompts and system instructions as versioned, reviewable engineering artifacts.
• Proven ability to ensure prompt strategies support determinism, reproducibility, and traceability in regulated environments (e.g., seeded examples, constrained decoding, output schemas, canonical evaluation sets).
• Experience overseeing prompt libraries and reusable patterns aligned with enterprise coding and architectural standards, including shared retrieval components and guardrail policies.
• A continuous learning mindset regarding new developments in Agentic AI and LLM-driven software coding.
• Experience with UI development frameworks (though not explicitly detailed, "UI" is in the title).

📝 Enhancement Note: The "2+ years applied experience" combined with "minimum 2 years applying them to LLM-enabled systems" suggests a specific requirement for practical, hands-on experience with LLMs in a professional setting, not just academic knowledge. The mention of "UI" in the title, though not detailed in the responsibilities or requirements, is a key skill to highlight as preferred. The "regulated environments" and "financial services" context is critical for understanding the depth of compliance and security expected.

📊 Process & Systems Portfolio Requirements

Portfolio Essentials:

• Demonstrate successful implementation of LLM-assisted software solutions, showcasing problem-solving capabilities in areas like RAG, agent workflows, or structured extraction.
• Provide examples of code quality, efficiency, and adherence to software engineering best practices in Java or Python, particularly within microservices or inference pipelines.
• Showcase experience with AWS cloud services and data management, including integration with managed model endpoints or vector services.
• Present evidence of implementing data quality rules, guardrails, and security protocols (e.g., PII redaction, access controls) in a regulated environment.

Process Documentation:

• Illustrate experience with CI/CD pipelines and agile methodologies, specifically highlighting LLMOps extensions such as prompt versioning, automated evaluations, and automated testing.
• Include documentation or architectural diagrams of LLM systems, detailing components like retrieval layers, vector stores, caching strategies, and observability setups.
• Provide examples of system design artifacts, model cards, or RAG/agent reference architectures that demonstrate an understanding of system constraints and regulatory expectations.
• Showcase contributions to prompt libraries, versioning strategies, or evaluation plans that ensure determinism, reproducibility, and traceability.

📝 Enhancement Note: Given the role's focus on LLMs and regulated environments, a portfolio should strongly emphasize demonstrable experience in building robust, secure, and compliant AI-driven software. Case studies showing the application of LLMs to solve specific business problems in finance, along with clear documentation of processes and results, will be highly valued.

💵 Compensation & Benefits

Salary Range:

• Based on the provided information (Software Engineer II, Bengaluru, India, 2-5 years experience, and JPMorgan Chase & Co. as a major financial institution), a competitive salary range for this role in Bengaluru can be estimated.
• Estimated Range: ₹15,00,000 to ₹28,00,000 per annum.
• Methodology: This estimate is derived from analyzing salary data for mid-level Software Engineers with Java/Python and cloud skills in Bengaluru, India, adjusted upwards for the specialized LLM expertise and the prestige of JPMorgan Chase. Data sources like Glassdoor, LinkedIn Salary, and industry reports for the Indian tech market were consulted. The range accounts for variations in candidate experience, specific skill depth, and the overall demand for LLM expertise.

Benefits:

• Comprehensive health insurance (medical, dental, vision).
• Retirement savings plans (e.g., Provident Fund, Gratuity as per Indian regulations).
• Paid time off (vacation, sick leave, public holidays).
• Professional development opportunities, including training, certifications, and conferences, especially in emerging AI/LLM technologies.
• Employee assistance programs for well-being.
• Potential for performance-based bonuses.
• Access to company-provided resources and tools for development and innovation.
• Opportunities for internal mobility and career advancement within a global organization.

Working Hours:

• Standard full-time working hours are typically 40 hours per week.
• Flexibility may be available, but the role requires on-site presence and potential on-call duties for Level 3 support.

📝 Enhancement Note: The salary range is an estimate for the Bengaluru market for a mid-level engineer with specialized AI skills at a major financial institution. Actual compensation will depend on the candidate's specific experience, interview performance, and the company's compensation bands. The benefits are typical for large multinational corporations in India.

🎯 Team & Company Context

🏢 Company Culture

Industry: Financial Services (Banking, Investment Banking, Technology Services) Company Size: JPMorgan Chase & Co. is a global financial services firm with over 300,000 employees worldwide. Founded: 2000 (merger of Chase Manhattan Corporation and J.P. Morgan & Co.)

Team Structure:

• The role is within the Commercial & Investment Bank’s Markets Tech Team.
• This team likely comprises software engineers, product managers, QA specialists, and potentially data scientists or AI specialists, working in an agile framework.
• Reporting structure will likely involve a Team Lead or Engineering Manager, with potential collaboration across different functional groups within Markets Tech and broader Technology divisions.

Methodology:

• Agile development methodologies are standard, with a focus on iterative development, continuous integration, and continuous delivery (CI/CD).
• The team employs LLM-specific methodologies, including LLMOps, prompt engineering best practices, and rigorous testing for AI components.
• Data-driven decision-making is central, utilizing telemetry, analytics, and performance metrics to guide improvements and maintain system stability.
• A strong emphasis is placed on security, compliance, and risk management due to the financial services context.

Company Website: https://www.jpmorganchase.com/

📝 Enhancement Note: JPMorgan Chase is a highly reputable, large-scale global financial institution. Its culture is typically characterized by a strong emphasis on performance, innovation within regulated boundaries, risk management, and career development. The Markets Tech Team specifically will be fast-paced and focused on delivering cutting-edge technology solutions for complex financial operations.

📈 Career & Growth Analysis

Operations Career Level: Software Engineer II is a mid-level role, typically requiring 2-5 years of relevant experience. It signifies a transition from foundational engineering tasks to taking ownership of significant components and contributing to system design.

Reporting Structure: The role reports into a Team Lead or Engineering Manager within the Markets Tech division. Collaboration will be expected with product owners, architects, other engineers, and potentially business stakeholders within the Commercial & Investment Bank.

Operations Impact: This role directly impacts the efficiency, reliability, and innovation of financial market technologies by integrating advanced LLM capabilities. Successful implementation of LLM solutions can lead to improved trading strategies, enhanced risk management, better data analysis, and streamlined operational processes, directly contributing to the firm's competitive edge and revenue generation.

Growth Opportunities:

• Technical Specialization: Deepen expertise in LLMs, AI/ML integration, cloud-native architectures (AWS), and specific financial technologies.
• Leadership Track: Progress to Senior Software Engineer, Staff Engineer, or Technical Lead roles, taking on more complex projects and mentoring junior engineers.
• Cross-Functional Roles: Opportunities to move into areas like AI/ML Engineering, Data Science, Cloud Architecture, or even Product Management within the financial technology domain.
• Management Track: With demonstrated leadership and people management skills, progression to Engineering Manager or similar leadership positions is possible.
• Continuous Learning: JPMorgan Chase invests heavily in training and development, offering access to internal courses, external certifications, and conferences to stay abreast of rapidly evolving fields like LLMs and AI.

📝 Enhancement Note: The growth path for a Software Engineer II at a firm like JPMorgan Chase is well-defined, with clear opportunities for both deep technical specialization and leadership. The company's investment in emerging technologies like LLMs suggests exciting future career possibilities within the organization.

🌐 Work Environment

Office Type: The role is on-site, indicating a traditional office-based work environment within JPMorgan Chase's Bengaluru campus. Office Location(s): The primary location is Bengaluru, India, specifically at the address provided: GR. FLR., 1ST TO 6TH FLR., PLATINA, BLOCK-3, KODBISANHALLI, OUTER RING , ROAD, BANGALORE EAST TAL., PIN 560103.

Workspace Context:

• Expect a collaborative office environment typical of a large tech organization, with shared workspaces, meeting rooms, and dedicated areas for focused work.
• Access to high-performance computing resources, modern development tools, and robust IT infrastructure will be provided.
• Opportunities for direct interaction and knowledge sharing with a diverse team of highly skilled engineers and subject matter experts in finance and AI.

Work Schedule:

• Standard working hours are expected to be around 40 hours per week.
• While an on-site role, there may be some flexibility depending on team norms and manager discretion.
• The requirement for Level 3 support may necessitate on-call rotations, which could involve working outside standard hours on an as-needed basis.

📝 Enhancement Note: The on-site requirement suggests a desire for in-person collaboration, team synergy, and direct access to company resources and infrastructure, which is common for roles involving sensitive financial data and complex, integrated systems.

📄 Application & Portfolio Review Process

Interview Process:

• Initial Screening: A recruiter or hiring manager will likely conduct an initial phone screen to assess basic qualifications, experience, and cultural fit.
• Technical Assessments: Expect one or more rounds of technical interviews. These may include:
  • Coding challenges (live coding or take-home assignments) focusing on Java/Python, algorithms, data structures, and problem-solving.
  • System design discussions, particularly around designing LLM-enabled systems, microservices, and cloud architectures on AWS.
  • LLM-specific questions covering concepts like RAG, prompt engineering, model limitations, and LLMOps.
• Behavioral Interviews: Questions assessing problem-solving approaches, teamwork, communication skills, and alignment with JPMorgan Chase's values.
• Manager/Team Interviews: Discussions with the hiring manager and potential team members to evaluate technical depth, team fit, and understanding of the role's responsibilities.
• Final Round: A final interview, potentially with a more senior leader, to confirm suitability and discuss career aspirations.

Portfolio Review Tips:

• Curate Select Projects: Focus on 2-3 high-impact projects that best showcase your LLM, Java/Python, AWS, and problem-solving skills.
• Highlight LLM Integration: Clearly explain the LLM's role in each project, the challenges you faced, and the solutions you implemented (e.g., RAG, prompt engineering, guardrails).
• Demonstrate Process & Metrics: For each project, describe your development process (Agile, CI/CD, LLMOps), and quantify the impact using relevant metrics (e.g., performance improvements, cost savings, accuracy gains, latency reduction).
• Showcase Code Quality: Be prepared to walk through snippets of your code, explaining design choices, efficiency considerations, and security measures.
• Tailor to the Role: Emphasize projects that align with the responsibilities and requirements mentioned in the job description, especially those involving regulated environments or complex system design.

Challenge Preparation:

• Algorithm & Data Structure Practice: Sharpen your skills in common interview algorithms and data structures. Platforms like LeetCode, HackerRank, and AlgoExpert are useful.
• System Design Fundamentals: Study scalable system design principles, microservices architecture, and cloud design patterns (especially AWS).
• LLM Concepts: Deeply understand LLM basics, common LLM applications (RAG, agents), prompt engineering techniques, evaluation metrics, and LLMOps lifecycle.
• Java/Python Proficiency: Be ready to write clean, efficient, and idiomatic code in both languages.
• Behavioral Question Framework: Prepare answers using the STAR method (Situation, Task, Action, Result) for common behavioral questions.
• Company Research: Understand JPMorgan Chase's business, its role in financial markets, and its commitment to technology and innovation.

📝 Enhancement Note: The interview process will be rigorous, reflecting JPMorgan Chase's standards for engineering talent, especially in specialized areas like AI. A strong portfolio demonstrating practical application of LLMs in a professional context will be crucial for success.

🛠 Tools & Technology Stack

Primary Tools:

• Programming Languages: Java, Python, SQL.
• Cloud Platform: Amazon Web Services (AWS).
• LLM Frameworks/Libraries: Potentially libraries like LangChain, LlamaIndex, Hugging Face Transformers, or direct API integrations with models like OpenAI's GPT series, Anthropic's Claude, or other enterprise-approved LLMs.
• Microservices Frameworks: Spring Boot (for Java), Flask/Django (for Python).
• Containerization: Docker, Kubernetes (likely part of the infrastructure).

Analytics & Reporting:

• Monitoring & Observability: Tools like Datadog, Prometheus, Grafana, CloudWatch for system health, performance, and LLM inference monitoring.
• Data Warehousing/Lakes: Redshift, Aurora, Databricks, or similar for storing and querying large datasets.
• BI Tools: Tableau, Power BI, or internal JPMorgan Chase tools for data visualization and reporting.

CRM & Automation:

• Version Control: Git, GitHub/GitLab/Bitbucket.
• CI/CD Tools: Jenkins, GitLab CI, AWS CodePipeline, or similar for automated builds, testing, and deployments.
• Orchestration/Workflow Management: Potentially tools for managing complex LLM agent workflows.
• Vector Databases: Pinecone, Weaviate, Milvus, or AWS-managed vector services for RAG implementations.

📝 Enhancement Note: The technology stack is a blend of enterprise-grade financial technology tools and cutting-edge AI/ML infrastructure. Proficiency in AWS, Java/Python, and a strong understanding of LLM-specific tools and concepts are paramount.

👥 Team Culture & Values

Operations Values:

• Integrity: Upholding the highest standards of ethical conduct and transparency, critical in financial services.
• Client Focus: Dedication to delivering value and solutions that meet the needs of internal business clients (Markets businesses).
• Innovation: Embracing new technologies like LLMs to drive efficiency and competitive advantage, while managing risks.
• Excellence: Striving for high performance, quality, and reliability in all aspects of software development and operations.
• Teamwork: Fostering a collaborative environment where diverse perspectives are valued and contributions are recognized.
• Accountability: Taking ownership of tasks, projects, and outcomes, especially in critical production systems.

Collaboration Style:

• Cross-Functional Integration: Strong emphasis on working closely with product managers, business analysts, QA engineers, and other engineering teams to deliver integrated solutions.
• Process-Oriented: Collaborative efforts to define, document, and optimize development and operational processes, especially those related to LLMOps and regulatory compliance.
• Knowledge Sharing: Encouraging open communication, code reviews, and sharing of best practices, particularly concerning emerging AI technologies and their application.
• Agile Principles: Adhering to collaborative agile ceremonies (stand-ups, sprint planning, retrospectives) to ensure efficient teamwork and continuous improvement.

📝 Enhancement Note: JPMorgan Chase's culture emphasizes a blend of rigorous execution, innovation, and strong ethical principles. For this role, collaboration will be key to successfully integrating novel AI technologies within a highly regulated and performance-driven environment.

⚡ Challenges & Growth Opportunities

Challenges:

• Integrating LLMs in Regulated Environments: Balancing the rapid evolution and potential unpredictability of LLMs with the stringent compliance, security, and risk management requirements of financial services.
• Ensuring Reliability and Determinism: Achieving consistent, predictable, and error-free outputs from LLMs, especially for critical business functions, requires advanced LLMOps and careful system design.
• Managing LLM Costs and Latency: Optimizing inference costs and ensuring low latency for LLM-powered applications can be technically challenging and requires efficient resource management.
• Keeping Pace with AI Advancements: The field of AI and LLMs is evolving at an unprecedented pace, requiring continuous learning and adaptation to new models, techniques, and tools.
• Bridging Technical and Business Needs: Effectively translating complex business requirements into technical specifications for LLM solutions and communicating technical trade-offs to non-technical stakeholders.

Learning & Development Opportunities:

• Specialized AI/ML Training: Access to internal and external training programs focused on advanced LLM techniques, prompt engineering, RAG, agentic AI, and MLOps.
• Cloud Certifications: Opportunities to obtain AWS certifications, deepening expertise in cloud-native development and deployment.
• Industry Conferences: Participation in leading AI, software engineering, and financial technology conferences to gain insights into the latest trends and network with peers.
• Mentorship Programs: Access to mentorship from senior engineers and AI specialists within JPMorgan Chase, providing guidance on career growth and technical development.
• Exposure to Diverse Projects: Working on a variety of LLM-powered applications across different business units within the Commercial & Investment Bank, offering broad exposure and skill development.

📝 Enhancement Note: The challenges inherent in this role are significant but also present excellent opportunities for deep learning and professional growth in one of the most exciting and rapidly advancing fields in technology.

💡 Interview Preparation

Strategy Questions:

• "Describe a complex technical problem you solved using Java/Python and how you approached it. What was the outcome?" (Focus on problem-solving methodology, code quality, and impact.)
• "How would you design a system to leverage LLMs for real-time market sentiment analysis, ensuring reliability and compliance with financial regulations?" (Assess system design skills, LLM application, and regulatory awareness.)
• "Explain the concept of Retrieval-Augmented Generation (RAG) and how you would implement it for a financial data retrieval application. What are the key challenges?" (Test understanding of core LLM techniques and practical implementation hurdles.)
• "Discuss your experience with LLMOps. How do you ensure reproducibility, versioning, and automated testing for LLM components?" (Evaluate practical experience with operationalizing LLMs.)

Company & Culture Questions:

• "Why are you interested in working at JPMorgan Chase, specifically within the Markets Tech Team?" (Assess research, alignment with company values, and career aspirations.)
• "Describe a time you had to collaborate with a difficult stakeholder or team member. How did you manage the situation?" (Evaluate teamwork, communication, and conflict resolution skills.)
• "How do you stay updated with the rapidly evolving field of AI and Large Language Models?" (Gauge commitment to continuous learning and industry awareness.)

Portfolio Presentation Strategy:

• Structure: Organize your portfolio by project. For each project, clearly state the problem, your role, the technologies used (especially LLMs, Java/Python, AWS), your specific contributions, the challenges faced, and the quantifiable results.
• LLM Focus: For LLM projects, detail the specific LLM used, the prompt engineering strategies employed, any guardrails or data quality checks implemented, and how you measured performance (e.g., accuracy, relevance, latency).
• Technical Depth: Be prepared to dive deep into the code, architecture, and design decisions. Explain why you chose certain approaches and what trade-offs were made.
• Storytelling: Frame your projects as compelling stories of problem-solving and innovation, highlighting your impact and learning.
• Conciseness: Keep your presentation focused and to the point, respecting the interviewer's time. Be ready for follow-up questions.

📝 Enhancement Note: Preparation should focus on demonstrating a strong foundational understanding of software engineering principles, practical experience with LLMs and AWS, and the ability to apply these skills within the specific context of financial markets and regulatory compliance.

📌 Application Steps

To apply for this software engineering position:

• Submit your application through the provided link on the JPMorgan Chase careers portal.
• Resume Optimization: Tailor your resume to highlight your experience with Java, Python, AWS, and especially your work with LLMs, RAG, prompt engineering, and LLMOps. Quantify achievements with specific metrics whenever possible.
• Portfolio Preparation: Curate a portfolio (e.g., GitHub repository, personal website, or detailed project descriptions) that showcases your LLM-powered projects, code quality, and system design capabilities. Ensure it includes examples relevant to the job description's responsibilities.
• Interview Practice: Practice coding challenges, system design scenarios, and behavioral questions. Be ready to discuss your LLM projects in detail and explain your approach to building reliable and compliant AI solutions.
• Company Research: Familiarize yourself with JPMorgan Chase's mission, values, and its role in the financial technology landscape. Understand the importance of innovation within a regulated industry.

⚠️ Important Notice: This enhanced job description includes AI-generated insights and operations industry-standard assumptions. All details should be verified directly with the hiring organization before making application decisions.