Data Architecture Using TOGAF

5 Day(s)

🎯 LEARNING OBJECTIVES

Understand the role of Data Architecture in Enterprise Architecture.
Understand data models, data lifecycle, pipelines, and storage.
Apply basic data integration, security, quality, and governance principles.
Understand relational, NoSQL, cloud, and hybrid data architectures.
Model data architecture using ArchiMate.

🧠 PREREQUISITES

Basic Enterprise Architecture knowledge.
Basic database and SQL knowledge.
Understanding of application architecture and APIs.
Basic knowledge of data security and governance.

01-Introduction

A data architecture describes how data is managed during its journey, from collection through to transformation, distribution, and consumption. It sets the blueprint for data and the way it flows through data storage systems. It is foundational to data processing operations and (AI) applications.

In includes the following:

Data Mesh: a modern decentralized data architecture paradigm that treats data as a product, with domain-oriented ownership and self-serve infrastructure, especially useful in large, complex organizations. As an organization, you should have clear modeling of your data architecture, what databases you have, the relationship between them, and the hosting nodes.

Conceptual data models: Database level:

They are also referred to as domain models and offer a big-picture view of what the system will contain, how it will be organized, and which business rules are involved

Logical data models: Schema and relations level:

They are less abstract and provide greater detail about the concepts and relationships in the domain under consideration.
Integration between different data models

Physical data models: Entities level:

They are less abstract and provide greater detail about the concepts and relationships in the domain under consideration. One of several formal data modeling notation systems is followed. These indicate data attributes, such as data types and their corresponding lengths, and show the relationships among entities. Logical data models don’t specify any technical system requirements.

02-Data Pipeline

Data has a use case, since it represents a solution that has specific objectives, which pass through the following lifecycle:

A.Data Collection: How do we collect data?

User inputs
Integration landscape
Content Crawling

B.Date refinement: How do we refine data?

Anti-Corruption Layer

To keep a smooth running between different co-worker independent databases, an anti-corruption layer should take place
Integration consideration should take place

Data health checker

How do you incrementally check the integrity between your data
How do you report it
How do you resolve it
Business rules
Validation rules
Consistency check
Data quality techniques

C.Data engine: How is data stored?

Data format:
- Relational: like Postgresql, Oracle, SQL Server, and MySQL
- Non-Relational: like Mongodb, DynamoDB, and Oracle Nosql
- Hybrid model
Location
- Cloud
- On prime
- Hybrid mode
Architecture Model
- Single database
- Multiple distributed databases
- Containerized DB

D.Data Analysis: How data analyzed?

Data analysis:
- What are the expected tools, reports, and analytics types
- Or will design built-in business components in the application architecture phase
- How to support the decision-making using these data
ML/DL Models
Analysis types:
- Real-time: so it will need different tools
- Periodic: to consider the data warehouse design

E.Data delivery:

At this section, we discuss How is data accessed/delivered?

To access data models, we have different approaches:

Create views on the database levels, whatever the technical methodology, using views, functions, procedures, or synonyms
APIs: Which provide a communication layer over the database, to be exposed to the external parties.
Exported Format: like tab and comma delimiters
Web services
Replication Models

3.Data Security Model:

At this section, we discuss How is data secured?

Connection security: Protects data while travelling between applications, databases, APIs, and other systems using TLS, strong authentication, certificates, and secure network segmentation. TLS 1.2/1.3, certificate lifecycle management, and disabling weak protocols and cipher suites are common best practices.
Database segregation: Separates sensitive data or workloads to reduce the impact of unauthorized access. A separate database provides stronger isolation with independent credentials, access controls, and potentially infrastructure, while Virtual Private Database (VPD) provides logical row-level isolation within the same database based on the user's authorization context.
Application and database-layer security: Applies defense-in-depth by enforcing security at both layers. The application provides authentication, authorization, RBAC/ABAC, and business-level access control, while the database independently enforces users, roles, privileges, views, VPD, and auditing. This prevents the database from relying entirely on application security.

Encryption for sensitive data: Encrypts particularly sensitive fields such as personal, financial, or confidential information at the application or column level. This provides granular protection even when someone has database access, provided that encryption keys are properly protected, rotated, and separated from the encrypted data.
TDE: Transparent Data Encryption protects database data at rest by encrypting database files and supported backup media without requiring application changes. In Oracle, TDE is commonly used to protect data files and backups from unauthorized access to underlying storage. However, TDE does not replace authorization, TLS, or application/column-level encryption because authorized database sessions can access decrypted data.

4.Data Versioning

Backup and restore provides a separate recovery mechanism for business continuity. The primary database is backed up periodically to independent backup storage, allowing the organization to recover data after major failures, data corruption, accidental deletion, ransomware, or other incidents. Unlike replication, backups can provide historical recovery points, including point-in-time recovery.

Real-Time Replication

Real-time replication continuously replicates changes from the Primary Site to the DR Site, keeping the DR database closely synchronized with the production database. This significantly reduces the Recovery Point Objective (RPO) because only a small amount of data may be lost if the primary site fails. The DR environment can then be activated through a controlled failover process.

Combined Business Continuity

Using both mechanisms provides stronger business continuity than relying on either one alone. Real-time replication supports rapid recovery and low RPO during site-level failures, while backup and restore provides protection against logical corruption, accidental deletion, ransomware, and the need to recover historical data. Therefore, replication should not be considered a replacement for backup; they address different recovery scenarios.

Overall Flow

Normal Operation: Primary Site → Real-Time Replication → DR Site

Data Protection: Primary Database → Periodic Backup → Backup Storage

Recovery: Backup Storage → Restore → DR Database

Site Failure: Primary Site → Failover → DR Site

Together, these mechanisms establish multiple recovery paths and improve Business Continuity, RPO, RTO, and operational resilience.

5. Data Quality

Data model attributes:
Business Rules Management:
Anti-Corruption Layer:
Data Quality practices:
- Completeness
  - - For text values
    - For numeric
    - For dates
- Correctness
  - - Domains
    - Anomaly detection
- Consistency
  - - Cross models validation
    - Cross Modules validation

6. Data Retention

What is Data Retention?

Definition: Data retention is the set of policies, processes, and technical controls that define how long data is stored, where it is stored, and how it is securely deleted after its lifecycle ends.

Objectives:

Legal compliance (GDPR, HIPAA, PCI DSS, SOX, ISO 27001).
Security (prevent misuse of stale data).
Cost optimization (storage efficiency).
Risk reduction (less data = less exposure in breaches).
Solution Performance

Architecture Approach:

Well-defined data retention policy
Alignment with business
Scheduler agent implementation
Data retention procedures
Data Model rebuilder

Hot/Cold Data retention approach

A hot/cold data retention model keeps only the actively used operational data in the primary database for two years, while older transactional data and logs are migrated to a separate cold database. GoldenGate can be used to replicate and support the controlled migration of eligible data, while configuration data and master/setup data remain in the primary database and are not subject to the transactional retention deletion.

0–2 years: Keep operational transactional data and required logs in the primary/hot database.
After 2 years: Migrate eligible transactional data and logs to the cold database.
Configuration data: Remains in the primary database and is not deleted as part of the transaction-retention process.
Master/Setup data: Remains in the primary database because it is required for system configuration and reference.
GoldenGate: Provides controlled replication/data movement between the operational and cold databases while minimizing disruption to production.
Primary DB benefit: Smaller active dataset, which can improve query performance, maintenance, backup duration, and storage efficiency.
Cold DB benefit: Retains historical transactions and logs for reporting, audit, investigation, and regulatory retention without placing the same workload on the operational database.
Key principle: Retention applies to transactional and log data—not automatically to configuration or master/setup data.

Data Retention Vs Data Decommissioning

Data Retention: Defines how long data must be kept and where it should be stored according to business, legal, regulatory, and organizational policy. For example, operational transactions may remain in the Hot DB for 2 years and then move to Cold Storage.
Data Decommissioning: Defines when and how data, databases, or systems are permanently retired or removed after they are no longer required, subject to the applicable retention, legal-hold, security, and disposal policies.

In short: Retention = Keep according to policy → Decommissioning = Retire/Delete according to policy.

7. Data democratization

What is Data democratization?

Data democratization is the practice of making trusted, relevant, and governed data easily accessible to the people who need it, without requiring them to depend entirely on IT, database teams, or data specialists for every analysis or decision.

The goal is self-service access to data while maintaining security, privacy, quality, and governance.

Example:
Instead of a business manager asking the DBA every time for licensing transaction statistics, the organization provides a governed data platform or BI dashboard where the manager can independently access approved data and analyze:

Key principles

Accessibility — authorized users can easily find and access data.
Self-service — users can analyze data without constant IT intervention.
Data literacy — users understand how to interpret and use data correctly.
Governance — access, ownership, quality, privacy, and compliance remain controlled.
Single source of truth — users work with consistent, trusted definitions.
Security — RBAC, data masking, classification, and least-privilege access remain enforced.

Data Democratization

8. Data documentation

Well-defined objects
Well-defined data models details:
- Object type
- Attributes
- Data types
- Domain models
- Sample

9. Open data consideration

Open Data is the practice of making selected government or organizational data publicly available in a machine-readable, reusable, and accessible format, while protecting confidential, personal, security-sensitive, and legally restricted information. The objective is to enable transparency, innovation, research, and the development of new digital services.
Open Data Process
The process typically starts with identifying and classifying candidate datasets, followed by data quality assessment, privacy and security review, anonymization where required, metadata definition, and publication through an Open Data Portal or API. Published datasets should have clear ownership, definitions, update frequency, licensing, and data-quality information.
Open Data Architecture
A typical architecture separates operational data from the public data layer. Data is extracted from approved source systems into an Open Data repository or data platform, where it can be validated, transformed, anonymized, and published through datasets and APIs. This prevents direct exposure of production systems and provides better governance and performance control.

Governance and Security
Not all organizational data should become open data. Data should pass through classification, privacy, security, legal, and business-owner approval before publication. Personally identifiable information (PII), confidential information, security-sensitive data, and restricted government information should be excluded or appropriately anonymized according to applicable regulations and organizational policies.
Business Value
Open Data can increase government transparency, enable innovation, support research, improve citizen services, and create opportunities for private-sector applications. For example, traffic authorities can publish aggregated traffic volumes, road-network information, parking availability, or transportation statistics through datasets and APIs without exposing individual customer or vehicle information.
Overall Flow
Source Systems → Data Classification → Quality & Privacy Assessment → Transformation/Anonymization → Open Data Repository → Open Data Portal/APIs → Citizens, Businesses & Researchers

Tools

Enterprise Architect
Archimate
Microsoft SQL Server
Oracle DBMS