1.1 Compare Data Formats (XML, JSON, and YAML)
📚 Overview: XML, JSON, and YAML are three popular data serialization formats used for data exchange between applications and systems.
Format Comparison Diagram
graph TB
A[Data Serialization Formats] --> B[XML]
A --> C[JSON]
A --> D[YAML]
B --> B1[Extensible Markup Language]
B --> B2[Tag-based structure]
B --> B3[Schema validation]
B --> B4[Verbose syntax]
C --> C1[JavaScript Object Notation]
C --> C2[Key-value pairs]
C --> C3[Lightweight]
C --> C4[Web APIs friendly]
D --> D1[YAML Ain't Markup Language]
D --> D2[Indentation-based]
D --> D3[Human-readable]
D --> D4[Configuration files]
style A fill:#667eea,stroke:#764ba2,stroke-width:3px,color:#fff
style B fill:#ff6b6b,stroke:#c92a2a,color:#fff
style C fill:#51cf66,stroke:#2f9e44,color:#fff
style D fill:#339af0,stroke:#1864ab,color:#fff
Comparison Table
| Feature | XML | JSON | YAML |
|---|---|---|---|
| Readability | Moderate | Good | Excellent |
| Verbosity | High | Low | Very Low |
| Data Types | Text-based | String, Number, Boolean, Null, Array, Object | Rich types including dates |
| Comments | <!-- comment --> | Not supported | # comment |
| Use Cases | SOAP APIs, Config files | REST APIs, Web apps | Configuration, Ansible, K8s |
| File Extension | .xml | .json | .yaml, .yml |
Example Representations
XML
<?xml version="1.0"?>
<person>
<name>John Doe</name>
<age>30</age>
<city>New York</city>
<skills>
<skill>Python</skill>
<skill>DevOps</skill>
</skills>
</person>
- ✓ Schema validation
- ✓ Attributes support
- ✗ Verbose
- ✗ Large file size
JSON
{
"name": "John Doe",
"age": 30,
"city": "New York",
"skills": [
"Python",
"DevOps"
]
}
- ✓ Lightweight
- ✓ Easy to parse
- ✓ Web-friendly
- ✗ No comments
YAML
name: John Doe age: 30 city: New York skills: - Python - DevOps
- ✓ Most readable
- ✓ Comments support
- ✓ Minimal syntax
- ✗ Whitespace sensitive
💡 Best Practices:
- Use JSON for REST APIs and web services
- Use YAML for configuration files and Infrastructure as Code
- Use XML for legacy systems and when schema validation is critical
1.2 Parsing Data Formats to Python Data Structures
🔄 Overview: Python provides built-in and third-party libraries to parse XML, JSON, and YAML into native Python data structures (dictionaries, lists, strings, etc.).
Parsing Flow Diagram
graph LR
A[Raw Data String/File] --> B{Format Type?}
B -->|XML| C[xml.etree.ElementTree]
B -->|JSON| D[json module]
B -->|YAML| E[PyYAML library]
C --> F[XML Tree Object]
D --> G[Python Dict/List]
E --> H[Python Dict/List]
F --> I[Access via methods]
G --> J[Access via keys]
H --> K[Access via keys]
style A fill:#667eea,stroke:#764ba2,color:#fff
style B fill:#ffd43b,stroke:#fab005,color:#000
style C fill:#ff6b6b,stroke:#c92a2a,color:#fff
style D fill:#51cf66,stroke:#2f9e44,color:#fff
style E fill:#339af0,stroke:#1864ab,color:#fff
JSON Parsing
sequenceDiagram
participant App as Python Application
participant JSON as json Module
participant Data as Python Dict/List
App->>JSON: json.loads(json_string)
JSON->>Data: Parse & Convert
Data-->>App: Return dict/list
App->>JSON: json.load(file_object)
JSON->>Data: Parse from file
Data-->>App: Return dict/list
Note over App,Data: Serialization (Python → JSON)
App->>JSON: json.dumps(python_obj)
JSON-->>App: Return JSON string
App->>JSON: json.dump(python_obj, file)
JSON-->>App: Write to file
# JSON Parsing in Python
import json
# Parse JSON string to Python dictionary
json_string = '{"name": "Alice", "age": 25, "skills": ["Python", "APIs"]}'
data = json.loads(json_string)
print(data['name']) # Output: Alice
print(data['skills'][0]) # Output: Python
# Parse JSON file
with open('data.json', 'r') as file:
data = json.load(file)
# Convert Python to JSON
python_dict = {"status": "success", "code": 200}
json_output = json.dumps(python_dict, indent=2)
print(json_output)
XML Parsing
# XML Parsing in Python
import xml.etree.ElementTree as ET
# Parse XML string
xml_string = '''
<?xml version="1.0"?>
<devices>
<device>
<name>Router1</name>
<ip>192.168.1.1</ip>
</device>
</devices>
'''
root = ET.fromstring(xml_string)
# Access elements
for device in root.findall('device'):
name = device.find('name').text
ip = device.find('ip').text
print(f"{name}: {ip}")
# Parse XML file
tree = ET.parse('devices.xml')
root = tree.getroot()
# Access with XPath-like syntax
devices = root.findall('.//device')
for device in devices:
print(device.attrib) # Get attributes
YAML Parsing
# YAML Parsing in Python
import yaml
# Parse YAML string
yaml_string = '''
server:
host: localhost
port: 8080
databases:
- name: users_db
type: postgresql
- name: cache_db
type: redis
'''
data = yaml.safe_load(yaml_string)
print(data['server']['host']) # Output: localhost
print(data['server']['databases'][0]['name']) # Output: users_db
# Parse YAML file
with open('config.yaml', 'r') as file:
config = yaml.safe_load(file)
# Convert Python to YAML
python_dict = {
'version': '1.0',
'services': ['web', 'api', 'database']
}
yaml_output = yaml.dump(python_dict, default_flow_style=False)
print(yaml_output)
Python Data Structure Mapping
graph TB
subgraph JSON/YAML
A1[Object/Mapping] --> P1[dict]
A2[Array/Sequence] --> P2[list]
A3[String] --> P3[str]
A4[Number] --> P4[int/float]
A5[Boolean] --> P5[bool]
A6[Null/None] --> P6[None]
end
subgraph XML
B1[Element] --> P7[Element Object]
B2[Text Content] --> P8[str]
B3[Attributes] --> P9[dict]
end
style P1 fill:#51cf66,stroke:#2f9e44,color:#fff
style P2 fill:#51cf66,stroke:#2f9e44,color:#fff
style P7 fill:#ff6b6b,stroke:#c92a2a,color:#fff
⚠️ Common Parsing Pitfalls:
- JSON: Keys must be strings, no trailing commas allowed
- XML: Namespaces can complicate parsing, use proper namespace handling
- YAML: Indentation errors will cause parsing failures, tabs are not allowed
- Always use
try-exceptblocks to handle parsing errors
1.3 Test-Driven Development (TDD)
🎯 Overview: Test-Driven Development is a software development methodology where tests are written before the actual code. It follows a "Red-Green-Refactor" cycle.
TDD Cycle Diagram
graph LR
A[Write Test] -->|Test Fails| B[Write Code]
B -->|Test Passes| C[Refactor]
C -->|Maintain Tests| A
style A fill:#ff6b6b,stroke:#c92a2a,stroke-width:3px,color:#fff
style B fill:#51cf66,stroke:#2f9e44,stroke-width:3px,color:#fff
style C fill:#339af0,stroke:#1864ab,stroke-width:3px,color:#fff
Detailed TDD Workflow
sequenceDiagram
participant Dev as Developer
participant Test as Test Suite
participant Code as Application Code
Note over Dev,Code: RED Phase
Dev->>Test: 1. Write failing test
Test->>Code: Run test
Code-->>Test: ❌ Test fails (expected)
Test-->>Dev: Failure feedback
Note over Dev,Code: GREEN Phase
Dev->>Code: 2. Write minimal code
Test->>Code: Run test again
Code-->>Test: ✅ Test passes
Test-->>Dev: Success feedback
Note over Dev,Code: REFACTOR Phase
Dev->>Code: 3. Improve code quality
Test->>Code: Run all tests
Code-->>Test: ✅ All tests still pass
Test-->>Dev: Confidence to refactor
TDD Principles
🔴 RED
Write a failing test
- Define expected behavior
- Test should fail initially
- Clarifies requirements
- Ensures test validity
🟢 GREEN
Write code to pass test
- Write minimal code
- Focus on functionality
- Make test pass quickly
- Don't optimize yet
🔵 REFACTOR
Improve code quality
- Clean up code
- Remove duplication
- Optimize performance
- Keep tests passing
TDD Example in Python
# Step 1: Write the test first (RED phase)
import unittest
class TestCalculator(unittest.TestCase):
def test_add_two_numbers(self):
calc = Calculator()
result = calc.add(2, 3)
self.assertEqual(result, 5)
def test_add_negative_numbers(self):
calc = Calculator()
result = calc.add(-1, -1)
self.assertEqual(result, -2)
# Running this test will FAIL because Calculator doesn't exist yet
# Step 2: Write minimal code to pass (GREEN phase)
class Calculator:
def add(self, a, b):
return a + b
# Now the tests PASS!
# Step 3: Refactor (REFACTOR phase)
class Calculator:
"""A simple calculator with basic operations."""
def add(self, a: float, b: float) -> float:
"""Add two numbers and return the result."""
if not isinstance(a, (int, float)) or not isinstance(b, (int, float)):
raise TypeError("Both arguments must be numbers")
return a + b
# Tests still PASS after refactoring!
Benefits of TDD
mindmap
root((TDD Benefits))
Code Quality
Better design
Modular code
Less complexity
Fewer bugs
Documentation
Tests as specs
Usage examples
Living documentation
Confidence
Safe refactoring
Regression prevention
Code coverage
Development
Faster debugging
Clear requirements
Incremental progress
Python Testing Frameworks
| Framework | Description | Key Features |
|---|---|---|
| unittest | Built-in Python testing framework | Standard library, xUnit-style, test discovery |
| pytest | Popular third-party framework | Simple syntax, fixtures, plugins, parametrization |
| doctest | Tests embedded in docstrings | Documentation + tests, simple examples |
| nose2 | Extends unittest | Test discovery, plugins, parallel execution |
Complete TDD Example: REST API Client
# test_api_client.py - Write tests first!
import unittest
from unittest.mock import patch, Mock
class TestAPIClient(unittest.TestCase):
def setUp(self):
self.client = APIClient("https://api.example.com")
@patch('requests.get')
def test_get_user_returns_user_data(self, mock_get):
# Arrange
mock_response = Mock()
mock_response.json.return_value = {"id": 1, "name": "Alice"}
mock_response.status_code = 200
mock_get.return_value = mock_response
# Act
result = self.client.get_user(1)
# Assert
self.assertEqual(result['name'], "Alice")
mock_get.assert_called_once_with("https://api.example.com/users/1")
@patch('requests.get')
def test_get_user_handles_404(self, mock_get):
# Arrange
mock_response = Mock()
mock_response.status_code = 404
mock_get.return_value = mock_response
# Act & Assert
with self.assertRaises(UserNotFoundException):
self.client.get_user(999)
# api_client.py - Now implement the code
import requests
class UserNotFoundException(Exception):
pass
class APIClient:
def __init__(self, base_url):
self.base_url = base_url
def get_user(self, user_id):
response = requests.get(f"{self.base_url}/users/{user_id}")
if response.status_code == 404:
raise UserNotFoundException(f"User {user_id} not found")
response.raise_for_status()
return response.json()
# Run tests: python -m unittest test_api_client.py
✅ TDD Best Practices:
- Write the smallest test possible
- One assertion per test (when practical)
- Test behavior, not implementation
- Keep tests fast and independent
- Use descriptive test names
- Follow AAA pattern: Arrange, Act, Assert
- Run tests frequently during development
⚠️ Common TDD Mistakes:
- Writing too many tests before implementing code
- Testing implementation details instead of behavior
- Skipping the refactor phase
- Writing tests that depend on each other
- Not running tests frequently enough
🐍 Interactive Python Interpreter
💻 Live Python Environment: Write and execute Python code directly in your browser! Try the examples below or write your own code to practice Python basics, data structures, functions, and classes.
Sample Code Templates
✏️ Python Code Editor
📤 Output
⏳ Loading Python interpreter...
✨ Features:
- Real-time Python execution in your browser
- Support for most standard Python 3 features
- Practice data structures (dictionaries, lists, tuples)
- Great for learning Python basics and DEVASC concepts
- No installation required - runs entirely in browser
⚠️ Limitations:
- Cannot import external libraries (json, requests, yaml, xml, etc.)
- No file system access (no open(), read(), write())
- Limited to basic Python features and some standard modules
- Some advanced Python features may not work
Quick Reference
Supported Modules
- ✓ math
- ✓ random
- ✓ time
- ✓ re (regex - basic)
Supported Features
- ✓ Lists, Dicts, Sets
- ✓ Functions & Classes
- ✓ Loops & Conditionals
- ✓ List Comprehensions
- ✓ Exception Handling
Keyboard Shortcuts
- Tab - Indent
- Shift+Tab - Unindent
- Ctrl+Enter - Run Code
- Ctrl+/ - Comment Line