Software Verification & Validation

Summary

Part 1: LangGraph Vague Spec Agent

A LangGraph agent classifies specifications as vague or precise, transforms vague ones, and generates test cases.

Agent Workflow

Five Test Inputs (All Expected: VAGUE)

#	Input Specification	Category	Why Vague
1	"The system shall allow for fast, easy data entry"	Software Dev	Undefined "fast" and "easy"
2	"Install high-quality flooring"	Construction	No material/brand/standard defined
3	"The application must be secure"	Performance	No security protocols/compliance specified
4	"The report will include, as appropriate, investigation findings"	Reporting	"As appropriate" leaves scope open
5	"The project will be completed in a timely manner"	Timeline	No specific deadline

File: part1_vague_spec_agent.py

Part 2A: Shopping Cart Testing

Step 1-2: Installation & Existing Test Execution

Repository: shopping-cart-unit-tests by AutomationPanda. Contains orders.py (60 stmts) with 29 existing tests.

$ python -m pytest tests/test_orders.py -v
29 passed in 0.10s

Step 3: Initial Statement Coverage

Missing lines 86-89: DynamicallyPricedItem.get_latest_price() (HTTP API call)

Step 4: Equivalence Partitioning — 20 Test Designs

Equivalence Classes

EC#	Source	Partition	Valid/Invalid
EC1	calculate_total.subtotal	subtotal = 0	Valid (boundary)
EC2	calculate_total.subtotal	subtotal > 0	Valid
EC3	calculate_total.subtotal	subtotal < 0	Invalid
EC4	calculate_total.shipping	shipping = 0	Valid (boundary)
EC5	calculate_total.shipping	shipping > 0	Valid
EC6	calculate_total.shipping	shipping < 0	Invalid
EC7	calculate_total.discount	discount = 0	Valid (boundary)
EC8	calculate_total.discount	0 < disc ≤ sub+ship	Valid
EC9	calculate_total.discount	disc > sub+ship	Valid (clamped)
EC10	calculate_total.discount	discount < 0	Invalid
EC11	calculate_total.tax_percent	tax = 0	Valid (boundary)
EC12	calculate_total.tax_percent	0 < tax < 1	Valid
EC13	calculate_total.tax_percent	tax < 0	Invalid
EC14	Item.unit_price	price = 0	Valid (boundary)
EC15	Item.unit_price	price > 0	Valid
EC16	Item.quantity	qty = 0	Valid (boundary)
EC17	Item.quantity	qty = 1 (default)	Valid
EC18	Item.quantity	qty > 1	Valid
EC19	Order.items	0 items (empty)	Valid (boundary)
EC20	Order.items	1 item	Valid
EC21	Order.items	multiple items	Valid
EC22	Order.reward	total < 1000	Valid (no bonus)
EC23	Order.reward	total ≥ 1000	Valid (bonus +10)
EC24	DynItem.quantity	qty = 1 (default)	Valid
EC25	DynItem.quantity	qty > 1	Valid

20 Functional Test Cases

TC#	Source .py	EP Classes	V/I?	Test Inputs	Expected Output	Status
TC01	orders.py	EC1,EC5,EC8,EC12	V	sub=0,ship=15,disc=5,tax=0.10	11.00	PASS
TC02	orders.py	EC2,EC4,EC7,EC12	V	sub=50,ship=0,disc=0,tax=0.08	54.00	PASS
TC03	orders.py	EC2,EC5,EC8,EC11	V	sub=100,ship=10,disc=20,tax=0	90.00	PASS
TC04	orders.py	EC2,EC5,EC9,EC12	V	sub=20,ship=5,disc=50,tax=0.10	0.00	PASS
TC05	orders.py	EC3	I	sub=-10,ship=5,disc=3,tax=0.05	ValueError	PASS
TC06	orders.py	EC6	I	sub=50,ship=-5,disc=3,tax=0.05	ValueError	PASS
TC07	orders.py	EC10	I	sub=50,ship=5,disc=-10,tax=0.05	ValueError	PASS
TC08	orders.py	EC13	I	sub=50,ship=5,disc=3,tax=-0.05	ValueError	PASS
TC09	orders.py	EC1,EC4,EC7,EC11	V	sub=0,ship=0,disc=0,tax=0	0.00	PASS
TC10	orders.py	EC2,EC5,EC8,EC12	V	sub=10000,ship=500,disc=1000,tax=0.15	10925.00	PASS
TC11	orders.py	EC14,EC18	V	Item("free",0,5)	total=0	PASS
TC12	orders.py	EC15,EC17	V	Item("book",29.99)	total=29.99	PASS
TC13	orders.py	EC15,EC18	V	Item("pen",1.50,10)	total=15.00	PASS
TC14	orders.py	EC15,EC16	V	Item("eraser",2.50,0)	total=0	PASS
TC15	orders.py	EC19	V	Order(5,0,0.10), no items	sub=0,total=5.50	PASS
TC16	orders.py	EC20	V	Order(10,5,0.07)+Item(25,2)	sub=50,total=58.85	PASS
TC17	orders.py	EC21	V	Order(0,0,0.05)+3 items	sub=10,total=10.50	PASS
TC18	orders.py	EC22	V	Order+Item(99.99,1)	points=99	PASS
TC19	orders.py	EC23	V	Order+Item(1000,1)	points=1010	PASS
TC20	orders.py	EC24,EC25	V	DynItem(101)+DynItem(202,4)	49.99, 40.00	PASS

Steps 5-8: Execution & Coverage Progression

Step 5: EP Tests — 20/20 Passed

$ python -m pytest tests/test_ep_functional.py -v
20 passed in 0.04s

Step 6: Coverage After EP Tests

EP tests exercise existing code paths — the 4 uncovered lines (86-89) require HTTP mock.

Step 7: Structural Tests Added

5 structural tests mock requests.get to exercise get_latest_price().

$ python -m pytest tests/test_structural.py -v
5 passed in 0.03s

Step 8: Final Coverage

Name        Stmts   Miss  Branch  BrPart  Cover
-------------------------------------------------
orders.py      60      0      14       0   100%
-------------------------------------------------
54 passed in 0.10s

Coverage Progression

Stage	Tests	Statement Coverage	Missing Lines
Original tests only	29	93%	86-89
+ 20 EP functional tests	49	93%	86-89
+ 5 structural tests	54	100%	None

Part 2B: LangSmith Tracing

The Quiz 1 agent was implemented with LangSmith tracing via @traceable decorators and environment configuration.

Expected Chains per Execution (Vague Input)

Chain	Type	LLM Calls
quiz1_vague_spec_pipeline	Top-level pipeline	—
classify_specification	Node	1
transform_to_precise	Node	1
generate_test_case	Node	1
Total		3

File: part2b_langsmith_tracing.py

Deliverable Files