wowdespacito/anthropics_skills
1
0
Fork 0
mirror of https://github.com/anthropics/skills.git synced 2026-08-02 13:05:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update docx, pptx, and xlsx skills (#1447)

Browse Source 
Add support for template formats (.dotx, .potx, .xltx) across validation
and the helper scripts.

Consolidate the shared office helpers into a single module, replacing the
pack/unpack pipeline with explicit zip/unzip steps. Extraction now rejects
symlink and path-traversal archive entries. Move run merging to a
standalone docx script, since it only ever applied to Word documents.

Fix the redlining validator so it compares against the original even when
a document has no tracked changes, which is when an untracked edit would
otherwise go unreported.

Provision a LibreOffice user profile per invocation so conversions work in
sandboxed environments.

Trim the skill docs to the guidance that earns its place.
This commit is contained in:
Peter Lai
2026-07-16 19:47:37 -07:00
committed by GitHub
parent 9d2f1ae187
commit fa0fa64bdc
53 changed files with 4147 additions and 4442 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/docx/SKILL.md
+65 -564
View File
@@ -1,590 +1,91 @@
--- ---
name: docx name: docx
description: "Use this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation." description: "Use this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Word templates (.dotx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation."
license: Proprietary. LICENSE.txt has complete terms license: Proprietary. LICENSE.txt has complete terms
--- ---
# DOCX creation, editing, and analysis # DOCX creation, editing, and analysis
## Overview A `.docx` is a ZIP archive of XML files. Choose your approach by task:
A .docx file is a ZIP archive containing XML files.
## Quick Reference
| Task | Approach | | Task | Approach |
|------|----------| |---|---|
| Read/analyze content | `pandoc` or unpack for raw XML | | **Create** a new document | Write a `docx` (npm) script — see gotchas below |
| Create new document | Use `docx-js` - see Creating New Documents below | | **Edit** an existing document | `unzip` → edit `word/document.xml` → `zip` (docx-js cannot open existing files) |
| Edit existing document | Unpack → edit XML → repack - see Editing Existing Documents below | | **Read** content | `pandoc -t markdown file.docx` |
### Converting .doc to .docx > Script paths below are relative to this skill's directory.
Legacy `.doc` files must be converted before editing: ## Creating with docx-js — gotchas
`docx` is preinstalled — do not run `npm install` first; write the script and `require('docx')` directly. Only if that require fails: `npm install docx`. The model knows the API; these are the footguns:
- **Page size defaults to A4.** For US Letter set `page: { size: { width: 12240, height: 15840 } }` (DXA; 1440 = 1″).
- **Landscape:** pass portrait dimensions and `orientation: PageOrientation.LANDSCAPE` — docx-js swaps width/height internally.
- **Tables need dual widths:** set `columnWidths` on the table AND `width` on every cell, both in `WidthType.DXA` (PERCENTAGE breaks in Google Docs). Column widths must sum to the table width.
- **Table shading:** use `ShadingType.CLEAR`, never `SOLID` (renders black).
- **Lists:** never insert `•` literally; use a `numbering` config with `LevelFormat.BULLET`.
- **`ImageRun` requires `type:`** (`"png"`, `"jpg"`, …).
- **`PageBreak` must be inside a `Paragraph`.**
- **Never use `\n`** — use separate `Paragraph` elements.
- **TOC:** headings must use built-in `HeadingLevel.*`; custom heading styles need `outlineLevel` set or they won't appear.
- **Don't use a table as a horizontal rule** — use a paragraph bottom border instead.
- **Dot-leader / right-aligned-on-same-line:** use `PositionalTab` (`alignment: PositionalTabAlignment.RIGHT`, `leader: PositionalTabLeader.DOT`) inside a `TextRun`, not literal `.` or space padding.
## Verify the output
After writing a `.docx`, render it and look at it:
```bash ```bash
python scripts/office/soffice.py --headless --convert-to docx document.doc python scripts/office/soffice.py --headless --convert-to pdf output.docx
pdftoppm -jpeg -r 100 output.pdf page
ls page-*.jpg # then Read the images
``` ```
### Reading Content `pdftoppm` zero-pads page numbers to the width of the page count (`page-01.jpg`…`page-12.jpg`).
## Editing existing documents
Legacy `.doc` files must be converted first: `python scripts/office/soffice.py --headless --convert-to docx file.doc`.
```bash ```bash
# Text extraction with tracked changes unzip -q doc.docx -d unpacked/
pandoc --track-changes=all document.docx -o output.md find unpacked -type l -delete # strip symlink entries — docx from external parties is untrusted
python scripts/merge_runs.py unpacked/ # coalesce fragmented runs so text is findable
# Raw XML access # edit unpacked/word/document.xml in place — do NOT reformat or pretty-print
python scripts/office/unpack.py document.docx unpacked/ (cd unpacked && rm -f ../out.docx && zip -Xr ../out.docx .)
python scripts/office/validate.py out.docx --original doc.docx # XSD checks; --auto-repair fixes common issues
# redlining? add --author "<the name you redlined under>" to check every edit is tracked
``` ```
### Converting to Images Word splits text across many `<w:r>` runs (revision ids, spell-check markers), so a phrase you can see in the document often doesn't exist as a contiguous string in the XML. `merge_runs.py` merges adjacent identically-formatted runs in `word/document.xml` without changing content or rendering; it also accepts a `.docx` directly (`python scripts/merge_runs.py doc.docx -o merged.docx`).
**Tracked changes:** when redlining, validate with `--author "<the name you redlined under>"` (needs `--original`) — it reports any text you changed without a `<w:ins>`/`<w:del>` around it, which is easy to do by accident and invisible in the accepted view. Wrap runs in `<w:ins>`/`<w:del>` with `w:id`, `w:author`, `w:date` attributes. Inside `<w:del>`, the text element is `<w:delText>`, not `<w:t>`. A deleted paragraph mark (`<w:pPr><w:rPr><w:del w:id=".." w:author=".." w:date=".."/></w:rPr></w:pPr>`) means "merge this paragraph into the next" — so deleting a paragraph outright is that plus a `<w:del>` around every run. The `<w:del/>` must come before the rPr's other children; their order is schema-enforced.
To produce a clean copy with all tracked changes accepted: `python scripts/accept_changes.py in.docx out.docx`.
Accepting a deleted paragraph mark should join that paragraph to the one below it, so a paragraph whose runs are *all* deleted vanishes. Word does this; `accept_changes.py` and `pandoc --track-changes=accept` don't always. Both fail the same way — they strip the deleted text but leave the emptied paragraph behind, which reads as a stray empty bullet when it was auto-numbered:
- `pandoc --track-changes=accept` never joins the paragraphs.
- `accept_changes.py` (LibreOffice) joins them correctly, except when the deleted paragraph is followed by an empty spacer paragraph.
An empty bullet in either view is an artifact of that view, not a defect in the document. Check paragraph deletions in the XML.
## Comments
Comments require six cross-linked files. Use the helper — directory mode when you'll also be editing `document.xml` (saves an unzip/rezip cycle), `.docx`-direct mode otherwise:
```bash ```bash
python scripts/office/soffice.py --headless --convert-to pdf document.docx # Against an already-unpacked directory (preferred when also placing markers)
pdftoppm -jpeg -r 150 document.pdf page python scripts/comment.py unpacked/ "Fees & expenses cap is too low"
python scripts/comment.py unpacked/ "Agreed" --parent 0
# Against a .docx directly
python scripts/comment.py contract.docx "This cap is too low" -o annotated.docx
``` ```
### Accepting Tracked Changes The script writes `comments.xml`, `commentsExtended.xml`, `commentsIds.xml`, `commentsExtensible.xml`, the relationships, and the content-type overrides. Comment IDs are auto-assigned. It then prints the `<w:commentRangeStart>`/`<w:commentRangeEnd>`/`<w:commentReference>` snippet to add to `word/document.xml` so the comment anchors to specific text — until you place those markers, the comment exists but is not visible.
To produce a clean document with all tracked changes accepted (requires LibreOffice):
```bash
python scripts/accept_changes.py input.docx output.docx
```
---
## Creating New Documents
Generate .docx files with JavaScript, then validate. Install: `npm install -g docx`
### Setup
```javascript
const { Document, Packer, Paragraph, TextRun, Table, TableRow, TableCell, ImageRun,
Header, Footer, AlignmentType, PageOrientation, LevelFormat, ExternalHyperlink,
InternalHyperlink, Bookmark, FootnoteReferenceRun, PositionalTab,
PositionalTabAlignment, PositionalTabRelativeTo, PositionalTabLeader,
TabStopType, TabStopPosition, Column, SectionType,
TableOfContents, HeadingLevel, BorderStyle, WidthType, ShadingType,
VerticalAlign, PageNumber, PageBreak } = require('docx');
const doc = new Document({ sections: [{ children: [/* content */] }] });
Packer.toBuffer(doc).then(buffer => fs.writeFileSync("doc.docx", buffer));
```
### Validation
After creating the file, validate it. If validation fails, unpack, fix the XML, and repack.
```bash
python scripts/office/validate.py doc.docx
```
### Page Size
```javascript
// CRITICAL: docx-js defaults to A4, not US Letter
// Always set page size explicitly for consistent results
sections: [{
properties: {
page: {
size: {
width: 12240, // 8.5 inches in DXA
height: 15840 // 11 inches in DXA
},
margin: { top: 1440, right: 1440, bottom: 1440, left: 1440 } // 1 inch margins
}
},
children: [/* content */]
}]
```
**Common page sizes (DXA units, 1440 DXA = 1 inch):**
| Paper | Width | Height | Content Width (1" margins) |
|-------|-------|--------|---------------------------|
| US Letter | 12,240 | 15,840 | 9,360 |
| A4 (default) | 11,906 | 16,838 | 9,026 |
**Landscape orientation:** docx-js swaps width/height internally, so pass portrait dimensions and let it handle the swap:
```javascript
size: {
width: 12240, // Pass SHORT edge as width
height: 15840, // Pass LONG edge as height
orientation: PageOrientation.LANDSCAPE // docx-js swaps them in the XML
},
// Content width = 15840 - left margin - right margin (uses the long edge)
```
### Styles (Override Built-in Headings)
Use Arial as the default font (universally supported). Keep titles black for readability.
```javascript
const doc = new Document({
styles: {
default: { document: { run: { font: "Arial", size: 24 } } }, // 12pt default
paragraphStyles: [
// IMPORTANT: Use exact IDs to override built-in styles
{ id: "Heading1", name: "Heading 1", basedOn: "Normal", next: "Normal", quickFormat: true,
run: { size: 32, bold: true, font: "Arial" },
paragraph: { spacing: { before: 240, after: 240 }, outlineLevel: 0 } }, // outlineLevel required for TOC
{ id: "Heading2", name: "Heading 2", basedOn: "Normal", next: "Normal", quickFormat: true,
run: { size: 28, bold: true, font: "Arial" },
paragraph: { spacing: { before: 180, after: 180 }, outlineLevel: 1 } },
]
},
sections: [{
children: [
new Paragraph({ heading: HeadingLevel.HEADING_1, children: [new TextRun("Title")] }),
]
}]
});
```
### Lists (NEVER use unicode bullets)
```javascript
// ❌ WRONG - never manually insert bullet characters
new Paragraph({ children: [new TextRun("• Item")] }) // BAD
new Paragraph({ children: [new TextRun("\u2022 Item")] }) // BAD
// ✅ CORRECT - use numbering config with LevelFormat.BULLET
const doc = new Document({
numbering: {
config: [
{ reference: "bullets",
levels: [{ level: 0, format: LevelFormat.BULLET, text: "•", alignment: AlignmentType.LEFT,
style: { paragraph: { indent: { left: 720, hanging: 360 } } } }] },
{ reference: "numbers",
levels: [{ level: 0, format: LevelFormat.DECIMAL, text: "%1.", alignment: AlignmentType.LEFT,
style: { paragraph: { indent: { left: 720, hanging: 360 } } } }] },
]
},
sections: [{
children: [
new Paragraph({ numbering: { reference: "bullets", level: 0 },
children: [new TextRun("Bullet item")] }),
new Paragraph({ numbering: { reference: "numbers", level: 0 },
children: [new TextRun("Numbered item")] }),
]
}]
});
// ⚠️ Each reference creates INDEPENDENT numbering
// Same reference = continues (1,2,3 then 4,5,6)
// Different reference = restarts (1,2,3 then 1,2,3)
```
### Tables
**CRITICAL: Tables need dual widths** - set both `columnWidths` on the table AND `width` on each cell. Without both, tables render incorrectly on some platforms.
```javascript
// CRITICAL: Always set table width for consistent rendering
// CRITICAL: Use ShadingType.CLEAR (not SOLID) to prevent black backgrounds
const border = { style: BorderStyle.SINGLE, size: 1, color: "CCCCCC" };
const borders = { top: border, bottom: border, left: border, right: border };
new Table({
width: { size: 9360, type: WidthType.DXA }, // Always use DXA (percentages break in Google Docs)
columnWidths: [4680, 4680], // Must sum to table width (DXA: 1440 = 1 inch)
rows: [
new TableRow({
children: [
new TableCell({
borders,
width: { size: 4680, type: WidthType.DXA }, // Also set on each cell
shading: { fill: "D5E8F0", type: ShadingType.CLEAR }, // CLEAR not SOLID
margins: { top: 80, bottom: 80, left: 120, right: 120 }, // Cell padding (internal, not added to width)
children: [new Paragraph({ children: [new TextRun("Cell")] })]
})
]
})
]
})
```
**Table width calculation:**
Always use `WidthType.DXA` — `WidthType.PERCENTAGE` breaks in Google Docs.
```javascript
// Table width = sum of columnWidths = content width
// US Letter with 1" margins: 12240 - 2880 = 9360 DXA
width: { size: 9360, type: WidthType.DXA },
columnWidths: [7000, 2360] // Must sum to table width
```
**Width rules:**
- **Always use `WidthType.DXA`** — never `WidthType.PERCENTAGE` (incompatible with Google Docs)
- Table width must equal the sum of `columnWidths`
- Cell `width` must match corresponding `columnWidth`
- Cell `margins` are internal padding - they reduce content area, not add to cell width
- For full-width tables: use content width (page width minus left and right margins)
### Images
```javascript
// CRITICAL: type parameter is REQUIRED
new Paragraph({
children: [new ImageRun({
type: "png", // Required: png, jpg, jpeg, gif, bmp, svg
data: fs.readFileSync("image.png"),
transformation: { width: 200, height: 150 },
altText: { title: "Title", description: "Desc", name: "Name" } // All three required
})]
})
```
### Page Breaks
```javascript
// CRITICAL: PageBreak must be inside a Paragraph
new Paragraph({ children: [new PageBreak()] })
// Or use pageBreakBefore
new Paragraph({ pageBreakBefore: true, children: [new TextRun("New page")] })
```
### Hyperlinks
```javascript
// External link
new Paragraph({
children: [new ExternalHyperlink({
children: [new TextRun({ text: "Click here", style: "Hyperlink" })],
link: "https://example.com",
})]
})
// Internal link (bookmark + reference)
// 1. Create bookmark at destination
new Paragraph({ heading: HeadingLevel.HEADING_1, children: [
new Bookmark({ id: "chapter1", children: [new TextRun("Chapter 1")] }),
]})
// 2. Link to it
new Paragraph({ children: [new InternalHyperlink({
children: [new TextRun({ text: "See Chapter 1", style: "Hyperlink" })],
anchor: "chapter1",
})]})
```
### Footnotes
```javascript
const doc = new Document({
footnotes: {
1: { children: [new Paragraph("Source: Annual Report 2024")] },
2: { children: [new Paragraph("See appendix for methodology")] },
},
sections: [{
children: [new Paragraph({
children: [
new TextRun("Revenue grew 15%"),
new FootnoteReferenceRun(1),
new TextRun(" using adjusted metrics"),
new FootnoteReferenceRun(2),
],
})]
}]
});
```
### Tab Stops
```javascript
// Right-align text on same line (e.g., date opposite a title)
new Paragraph({
children: [
new TextRun("Company Name"),
new TextRun("\tJanuary 2025"),
],
tabStops: [{ type: TabStopType.RIGHT, position: TabStopPosition.MAX }],
})
// Dot leader (e.g., TOC-style)
new Paragraph({
children: [
new TextRun("Introduction"),
new TextRun({ children: [
new PositionalTab({
alignment: PositionalTabAlignment.RIGHT,
relativeTo: PositionalTabRelativeTo.MARGIN,
leader: PositionalTabLeader.DOT,
}),
"3",
]}),
],
})
```
### Multi-Column Layouts
```javascript
// Equal-width columns
sections: [{
properties: {
column: {
count: 2, // number of columns
space: 720, // gap between columns in DXA (720 = 0.5 inch)
equalWidth: true,
separate: true, // vertical line between columns
},
},
children: [/* content flows naturally across columns */]
}]
// Custom-width columns (equalWidth must be false)
sections: [{
properties: {
column: {
equalWidth: false,
children: [
new Column({ width: 5400, space: 720 }),
new Column({ width: 3240 }),
],
},
},
children: [/* content */]
}]
```
Force a column break with a new section using `type: SectionType.NEXT_COLUMN`.
### Table of Contents
```javascript
// CRITICAL: Headings must use HeadingLevel ONLY - no custom styles
new TableOfContents("Table of Contents", { hyperlink: true, headingStyleRange: "1-3" })
```
### Headers/Footers
```javascript
sections: [{
properties: {
page: { margin: { top: 1440, right: 1440, bottom: 1440, left: 1440 } } // 1440 = 1 inch
},
headers: {
default: new Header({ children: [new Paragraph({ children: [new TextRun("Header")] })] })
},
footers: {
default: new Footer({ children: [new Paragraph({
children: [new TextRun("Page "), new TextRun({ children: [PageNumber.CURRENT] })]
})] })
},
children: [/* content */]
}]
```
### Critical Rules for docx-js
- **Set page size explicitly** - docx-js defaults to A4; use US Letter (12240 x 15840 DXA) for US documents
- **Landscape: pass portrait dimensions** - docx-js swaps width/height internally; pass short edge as `width`, long edge as `height`, and set `orientation: PageOrientation.LANDSCAPE`
- **Never use `\n`** - use separate Paragraph elements
- **Never use unicode bullets** - use `LevelFormat.BULLET` with numbering config
- **PageBreak must be in Paragraph** - standalone creates invalid XML
- **ImageRun requires `type`** - always specify png/jpg/etc
- **Always set table `width` with DXA** - never use `WidthType.PERCENTAGE` (breaks in Google Docs)
- **Tables need dual widths** - `columnWidths` array AND cell `width`, both must match
- **Table width = sum of columnWidths** - for DXA, ensure they add up exactly
- **Always add cell margins** - use `margins: { top: 80, bottom: 80, left: 120, right: 120 }` for readable padding
- **Use `ShadingType.CLEAR`** - never SOLID for table shading
- **Never use tables as dividers/rules** - cells have minimum height and render as empty boxes (including in headers/footers); use `border: { bottom: { style: BorderStyle.SINGLE, size: 6, color: "2E75B6", space: 1 } }` on a Paragraph instead. For two-column footers, use tab stops (see Tab Stops section), not tables
- **TOC requires HeadingLevel only** - no custom styles on heading paragraphs
- **Override built-in styles** - use exact IDs: "Heading1", "Heading2", etc.
- **Include `outlineLevel`** - required for TOC (0 for H1, 1 for H2, etc.)
---
## Editing Existing Documents
**Follow all 3 steps in order.**
### Step 1: Unpack
```bash
python scripts/office/unpack.py document.docx unpacked/
```
Extracts XML, pretty-prints, merges adjacent runs, and converts smart quotes to XML entities (`&#x201C;` etc.) so they survive editing. Use `--merge-runs false` to skip run merging.
### Step 2: Edit XML
Edit files in `unpacked/word/`. See XML Reference below for patterns.
**Use "Claude" as the author** for tracked changes and comments, unless the user explicitly requests use of a different name.
**Use the Edit tool directly for string replacement. Do not write Python scripts.** Scripts introduce unnecessary complexity. The Edit tool shows exactly what is being replaced.
**CRITICAL: Use smart quotes for new content.** When adding text with apostrophes or quotes, use XML entities to produce smart quotes:
```xml
<!-- Use these entities for professional typography -->
<w:t>Here&#x2019;s a quote: &#x201C;Hello&#x201D;</w:t>
```
| Entity | Character |
|--------|-----------|
| `&#x2018;` | ‘ (left single) |
| `&#x2019;` | ’ (right single / apostrophe) |
| `&#x201C;` | “ (left double) |
| `&#x201D;` | ” (right double) |
**Adding comments:** Use `comment.py` to handle boilerplate across multiple XML files (text must be pre-escaped XML):
```bash
python scripts/comment.py unpacked/ 0 "Comment text with &amp; and &#x2019;"
python scripts/comment.py unpacked/ 1 "Reply text" --parent 0 # reply to comment 0
python scripts/comment.py unpacked/ 0 "Text" --author "Custom Author" # custom author name
```
Then add markers to document.xml (see Comments in XML Reference).
### Step 3: Pack
```bash
python scripts/office/pack.py unpacked/ output.docx --original document.docx
```
Validates with auto-repair, condenses XML, and creates DOCX. Use `--validate false` to skip.
**Auto-repair will fix:**
- `durableId` >= 0x7FFFFFFF (regenerates valid ID)
- Missing `xml:space="preserve"` on `<w:t>` with whitespace
**Auto-repair won't fix:**
- Malformed XML, invalid element nesting, missing relationships, schema violations
### Common Pitfalls
- **Replace entire `<w:r>` elements**: When adding tracked changes, replace the whole `<w:r>...</w:r>` block with `<w:del>...<w:ins>...` as siblings. Don't inject tracked change tags inside a run.
- **Preserve `<w:rPr>` formatting**: Copy the original run's `<w:rPr>` block into your tracked change runs to maintain bold, font size, etc.
---
## XML Reference
### Schema Compliance
- **Element order in `<w:pPr>`**: `<w:pStyle>`, `<w:numPr>`, `<w:spacing>`, `<w:ind>`, `<w:jc>`, `<w:rPr>` last
- **Whitespace**: Add `xml:space="preserve"` to `<w:t>` with leading/trailing spaces
- **RSIDs**: Must be 8-digit hex (e.g., `00AB1234`)
### Tracked Changes
**Insertion:**
```xml
<w:ins w:id="1" w:author="Claude" w:date="2025-01-01T00:00:00Z">
<w:r><w:t>inserted text</w:t></w:r>
</w:ins>
```
**Deletion:**
```xml
<w:del w:id="2" w:author="Claude" w:date="2025-01-01T00:00:00Z">
<w:r><w:delText>deleted text</w:delText></w:r>
</w:del>
```
**Inside `<w:del>`**: Use `<w:delText>` instead of `<w:t>`, and `<w:delInstrText>` instead of `<w:instrText>`.
**Minimal edits** - only mark what changes:
```xml
<!-- Change "30 days" to "60 days" -->
<w:r><w:t>The term is </w:t></w:r>
<w:del w:id="1" w:author="Claude" w:date="...">
<w:r><w:delText>30</w:delText></w:r>
</w:del>
<w:ins w:id="2" w:author="Claude" w:date="...">
<w:r><w:t>60</w:t></w:r>
</w:ins>
<w:r><w:t> days.</w:t></w:r>
```
**Deleting entire paragraphs/list items** - when removing ALL content from a paragraph, also mark the paragraph mark as deleted so it merges with the next paragraph. Add `<w:del/>` inside `<w:pPr><w:rPr>`:
```xml
<w:p>
<w:pPr>
<w:numPr>...</w:numPr> <!-- list numbering if present -->
<w:rPr>
<w:del w:id="1" w:author="Claude" w:date="2025-01-01T00:00:00Z"/>
</w:rPr>
</w:pPr>
<w:del w:id="2" w:author="Claude" w:date="2025-01-01T00:00:00Z">
<w:r><w:delText>Entire paragraph content being deleted...</w:delText></w:r>
</w:del>
</w:p>
```
Without the `<w:del/>` in `<w:pPr><w:rPr>`, accepting changes leaves an empty paragraph/list item.
**Rejecting another author's insertion** - nest deletion inside their insertion:
```xml
<w:ins w:author="Jane" w:id="5">
<w:del w:author="Claude" w:id="10">
<w:r><w:delText>their inserted text</w:delText></w:r>
</w:del>
</w:ins>
```
**Restoring another author's deletion** - add insertion after (don't modify their deletion):
```xml
<w:del w:author="Jane" w:id="5">
<w:r><w:delText>deleted text</w:delText></w:r>
</w:del>
<w:ins w:author="Claude" w:id="10">
<w:r><w:t>deleted text</w:t></w:r>
</w:ins>
```
### Comments
After running `comment.py` (see Step 2), add markers to document.xml. For replies, use `--parent` flag and nest markers inside the parent's.
**CRITICAL: `<w:commentRangeStart>` and `<w:commentRangeEnd>` are siblings of `<w:r>`, never inside `<w:r>`.**
```xml
<!-- Comment markers are direct children of w:p, never inside w:r -->
<w:commentRangeStart w:id="0"/>
<w:del w:id="1" w:author="Claude" w:date="2025-01-01T00:00:00Z">
<w:r><w:delText>deleted</w:delText></w:r>
</w:del>
<w:r><w:t> more text</w:t></w:r>
<w:commentRangeEnd w:id="0"/>
<w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="0"/></w:r>
<!-- Comment 0 with reply 1 nested inside -->
<w:commentRangeStart w:id="0"/>
<w:commentRangeStart w:id="1"/>
<w:r><w:t>text</w:t></w:r>
<w:commentRangeEnd w:id="1"/>
<w:commentRangeEnd w:id="0"/>
<w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="0"/></w:r>
<w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="1"/></w:r>
```
### Images
1. Add image file to `word/media/`
2. Add relationship to `word/_rels/document.xml.rels`:
```xml
<Relationship Id="rId5" Type=".../image" Target="media/image1.png"/>
```
3. Add content type to `[Content_Types].xml`:
```xml
<Default Extension="png" ContentType="image/png"/>
```
4. Reference in document.xml:
```xml
<w:drawing>
<wp:inline>
<wp:extent cx="914400" cy="914400"/> <!-- EMUs: 914400 = 1 inch -->
<a:graphic>
<a:graphicData uri=".../picture">
<pic:pic>
<pic:blipFill><a:blip r:embed="rId5"/></pic:blipFill>
</pic:pic>
</a:graphicData>
</a:graphic>
</wp:inline>
</w:drawing>
```
---
## Dependencies ## Dependencies
- **pandoc**: Text extraction `docx` (npm, preinstalled — install only if `require('docx')` fails) · `pandoc` · LibreOffice (`soffice`) · `pdftoppm` (Poppler)
- **docx**: `npm install -g docx` (new documents)
- **LibreOffice**: PDF conversion (auto-configured for sandboxed environments via `scripts/office/soffice.py`)
- **Poppler**: `pdftoppm` for images
skills/docx/scripts/comment.py
+177 -127
View File
@@ -1,26 +1,41 @@
"""Add comments to DOCX documents. """Add comments to a DOCX document.
Accepts either an unpacked directory OR a .docx/.dotx file directly.
Usage: Usage:
python comment.py unpacked/ 0 "Comment text" # Against an unpacked directory (writes satellite files in place)
python comment.py unpacked/ 1 "Reply text" --parent 0 python comment.py unpacked/ "Comment text"
python comment.py unpacked/ "Reply text" --parent 0
Text should be pre-escaped XML (e.g., &amp; for &, &#x2019; for smart quotes). # Against a .docx directly (extracts, writes satellite files, rezips)
python comment.py contract.docx "This cap is too low" -o annotated.docx
python comment.py contract.docx "Comment" --id 5 # explicit ID
After running, add markers to document.xml: The comment ID is auto-assigned (max existing + 1) unless --id is given.
<w:commentRangeStart w:id="0"/> Plain text is XML-escaped automatically; if you pass already-escaped text
(e.g. &amp;, &#x2019;) use --raw to skip escaping.
After running, add markers to word/document.xml so the comment is visible:
<w:commentRangeStart w:id="N"/>
... commented content ... ... commented content ...
<w:commentRangeEnd w:id="0"/> <w:commentRangeEnd w:id="N"/>
<w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="0"/></w:r> <w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="N"/></w:r>
""" """
import argparse import argparse
import random import random
import shutil import shutil
import sys import sys
import tempfile
import zipfile
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
from xml.parsers.expat import ExpatError
from xml.sax.saxutils import escape as xml_escape
from office.helpers import opc_target, rezip as _rezip, safe_extract as _safe_extract
TEMPLATE_DIR = Path(__file__).parent / "templates" TEMPLATE_DIR = Path(__file__).parent / "templates"
NS = { NS = {
@@ -44,39 +59,38 @@ COMMENT_XML = """\
<w:sz w:val="20"/> <w:sz w:val="20"/>
<w:szCs w:val="20"/> <w:szCs w:val="20"/>
</w:rPr> </w:rPr>
<w:t>{text}</w:t> <w:t xml:space="preserve">{text}</w:t>
</w:r> </w:r>
</w:p> </w:p>
</w:comment>""" </w:comment>"""
COMMENT_MARKER_TEMPLATE = """ COMMENT_MARKER_TEMPLATE = """
Add to document.xml (markers must be direct children of w:p, never inside w:r): Add to word/document.xml (markers must be direct children of w:p, never inside w:r):
<w:commentRangeStart w:id="{cid}"/> <w:commentRangeStart w:id="{cid}"/>
<w:r>...</w:r> <w:r>...</w:r>
<w:commentRangeEnd w:id="{cid}"/> <w:commentRangeEnd w:id="{cid}"/>
<w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="{cid}"/></w:r>""" <w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="{cid}"/></w:r>"""
REPLY_MARKER_TEMPLATE = """ REPLY_MARKER_TEMPLATE = """
Nest markers inside parent {pid}'s markers (markers must be direct children of w:p, never inside w:r): Nest markers inside parent {pid}'s markers (direct children of w:p, never inside w:r):
<w:commentRangeStart w:id="{pid}"/><w:commentRangeStart w:id="{cid}"/> <w:commentRangeStart w:id="{pid}"/><w:commentRangeStart w:id="{cid}"/>
<w:r>...</w:r> <w:r>...</w:r>
<w:commentRangeEnd w:id="{cid}"/><w:commentRangeEnd w:id="{pid}"/> <w:commentRangeEnd w:id="{cid}"/><w:commentRangeEnd w:id="{pid}"/>
<w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="{pid}"/></w:r> <w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="{pid}"/></w:r>
<w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="{cid}"/></w:r>""" <w:r><w:rPr><w:rStyle w:val="CommentReference"/></w:rPr><w:commentReference w:id="{cid}"/></w:r>"""
SMART_QUOTE_ENTITIES = {
"“": "&#x201C;",
"”": "&#x201D;",
"‘": "&#x2018;",
"’": "&#x2019;",
}
def _generate_hex_id() -> str: def _generate_hex_id() -> str:
return f"{random.randint(0, 0x7FFFFFFE):08X}" return f"{random.randint(0, 0x7FFFFFFE):08X}"
SMART_QUOTE_ENTITIES = {
"\u201c": "&#x201C;",
"\u201d": "&#x201D;",
"\u2018": "&#x2018;",
"\u2019": "&#x2019;",
}
def _encode_smart_quotes(text: str) -> str: def _encode_smart_quotes(text: str) -> str:
for char, entity in SMART_QUOTE_ENTITIES.items(): for char, entity in SMART_QUOTE_ENTITIES.items():
text = text.replace(char, entity) text = text.replace(char, entity)
@@ -105,6 +119,19 @@ def _find_para_id(comments_path: Path, comment_id: int) -> str | None:
return None return None
def _next_comment_id(comments_path: Path) -> int:
if not comments_path.exists():
return 0
dom = defusedxml.minidom.parseString(comments_path.read_text(encoding="utf-8"))
ids = []
for c in dom.getElementsByTagName("w:comment"):
try:
ids.append(int(c.getAttribute("w:id")))
except ValueError:
pass
return (max(ids) + 1) if ids else 0
def _get_next_rid(rels_path: Path) -> int: def _get_next_rid(rels_path: Path) -> int:
dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8")) dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8"))
max_rid = 0 max_rid = 0
@@ -120,151 +147,146 @@ def _get_next_rid(rels_path: Path) -> int:
def _has_relationship(rels_path: Path, target: str) -> bool: def _has_relationship(rels_path: Path, target: str) -> bool:
dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8")) dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8"))
for rel in dom.getElementsByTagName("Relationship"): return any(
if rel.getAttribute("Target") == target: rel.getAttribute("Target") == target
return True for rel in dom.getElementsByTagName("Relationship")
return False )
def _has_content_type(ct_path: Path, part_name: str) -> bool: def _has_content_type(ct_path: Path, part_name: str) -> bool:
dom = defusedxml.minidom.parseString(ct_path.read_text(encoding="utf-8")) dom = defusedxml.minidom.parseString(ct_path.read_text(encoding="utf-8"))
for override in dom.getElementsByTagName("Override"): return any(
if override.getAttribute("PartName") == part_name: o.getAttribute("PartName") == part_name
return True for o in dom.getElementsByTagName("Override")
return False )
_COMMENT_RELS = [
("http://schemas.openxmlformats.org/officeDocument/2006/relationships/comments", "comments.xml"),
("http://schemas.microsoft.com/office/2011/relationships/commentsExtended", "commentsExtended.xml"),
("http://schemas.microsoft.com/office/2016/09/relationships/commentsIds", "commentsIds.xml"),
("http://schemas.microsoft.com/office/2018/08/relationships/commentsExtensible", "commentsExtensible.xml"),
]
_COMMENT_OVERRIDES = [
("/word/comments.xml", "application/vnd.openxmlformats-officedocument.wordprocessingml.comments+xml"),
("/word/commentsExtended.xml", "application/vnd.openxmlformats-officedocument.wordprocessingml.commentsExtended+xml"),
("/word/commentsIds.xml", "application/vnd.openxmlformats-officedocument.wordprocessingml.commentsIds+xml"),
("/word/commentsExtensible.xml", "application/vnd.openxmlformats-officedocument.wordprocessingml.commentsExtensible+xml"),
]
def _ensure_comment_relationships(unpacked_dir: Path) -> None: def _ensure_comment_relationships(unpacked_dir: Path) -> None:
rels_path = unpacked_dir / "word" / "_rels" / "document.xml.rels" rels_path = unpacked_dir / "word" / "_rels" / "document.xml.rels"
if not rels_path.exists(): if not rels_path.exists():
return return
if _has_relationship(rels_path, "comments.xml"):
return
dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8")) dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8"))
root = dom.documentElement root = dom.documentElement
comment_types = {rel_type for rel_type, _ in _COMMENT_RELS}
existing = set()
for rel in dom.getElementsByTagName("Relationship"):
if rel.getAttribute("Type") not in comment_types:
continue
part = opc_target(
rel.getAttribute("Target"),
"word/document.xml",
rel.getAttribute("TargetMode"),
)
if part is not None:
existing.add(part)
next_rid = _get_next_rid(rels_path) next_rid = _get_next_rid(rels_path)
changed = False
rels = [ for rel_type, target in _COMMENT_RELS:
( if opc_target(target, "word/document.xml") in existing:
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/comments", continue
"comments.xml",
),
(
"http://schemas.microsoft.com/office/2011/relationships/commentsExtended",
"commentsExtended.xml",
),
(
"http://schemas.microsoft.com/office/2016/09/relationships/commentsIds",
"commentsIds.xml",
),
(
"http://schemas.microsoft.com/office/2018/08/relationships/commentsExtensible",
"commentsExtensible.xml",
),
]
for rel_type, target in rels:
rel = dom.createElement("Relationship") rel = dom.createElement("Relationship")
rel.setAttribute("Id", f"rId{next_rid}") rel.setAttribute("Id", f"rId{next_rid}")
rel.setAttribute("Type", rel_type) rel.setAttribute("Type", rel_type)
rel.setAttribute("Target", target) rel.setAttribute("Target", target)
root.appendChild(rel) root.appendChild(rel)
next_rid += 1 next_rid += 1
changed = True
rels_path.write_bytes(dom.toxml(encoding="UTF-8")) if changed:
rels_path.write_bytes(dom.toxml(encoding="UTF-8"))
def _ensure_comment_content_types(unpacked_dir: Path) -> None: def _ensure_comment_content_types(unpacked_dir: Path) -> None:
ct_path = unpacked_dir / "[Content_Types].xml" ct_path = unpacked_dir / "[Content_Types].xml"
if not ct_path.exists(): if not ct_path.exists():
return return
if _has_content_type(ct_path, "/word/comments.xml"):
return
dom = defusedxml.minidom.parseString(ct_path.read_text(encoding="utf-8")) dom = defusedxml.minidom.parseString(ct_path.read_text(encoding="utf-8"))
root = dom.documentElement root = dom.documentElement
existing = {
overrides = [ o.getAttribute("PartName")
( for o in dom.getElementsByTagName("Override")
"/word/comments.xml", }
"application/vnd.openxmlformats-officedocument.wordprocessingml.comments+xml", changed = False
), for part_name, content_type in _COMMENT_OVERRIDES:
( if part_name in existing:
"/word/commentsExtended.xml", continue
"application/vnd.openxmlformats-officedocument.wordprocessingml.commentsExtended+xml",
),
(
"/word/commentsIds.xml",
"application/vnd.openxmlformats-officedocument.wordprocessingml.commentsIds+xml",
),
(
"/word/commentsExtensible.xml",
"application/vnd.openxmlformats-officedocument.wordprocessingml.commentsExtensible+xml",
),
]
for part_name, content_type in overrides:
override = dom.createElement("Override") override = dom.createElement("Override")
override.setAttribute("PartName", part_name) override.setAttribute("PartName", part_name)
override.setAttribute("ContentType", content_type) override.setAttribute("ContentType", content_type)
root.appendChild(override) root.appendChild(override)
changed = True
ct_path.write_bytes(dom.toxml(encoding="UTF-8")) if changed:
ct_path.write_bytes(dom.toxml(encoding="UTF-8"))
def add_comment( def add_comment(
unpacked_dir: str, unpacked_dir: Path | str,
comment_id: int,
text: str, text: str,
comment_id: int | None = None,
author: str = "Claude", author: str = "Claude",
initials: str = "C", initials: str = "C",
parent_id: int | None = None, parent_id: int | None = None,
) -> tuple[str, str]: raw: bool = False,
word = Path(unpacked_dir) / "word" ) -> tuple[int, str, str]:
unpacked_dir = Path(unpacked_dir)
if not raw:
text = xml_escape(text)
author = xml_escape(author, {'"': "&quot;"})
initials = xml_escape(initials, {'"': "&quot;"})
word = unpacked_dir / "word"
if not word.exists(): if not word.exists():
return "", f"Error: {word} not found" raise FileNotFoundError(f"{word} not found (not an unpacked .docx?)")
comments = word / "comments.xml"
if comment_id is None:
comment_id = _next_comment_id(comments)
parent_para = None
if parent_id is not None:
parent_para = _find_para_id(comments, parent_id) if comments.exists() else None
if not parent_para:
raise ValueError(f"parent comment {parent_id} not found")
para_id, durable_id = _generate_hex_id(), _generate_hex_id() para_id, durable_id = _generate_hex_id(), _generate_hex_id()
ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
comments = word / "comments.xml" if not comments.exists():
first_comment = not comments.exists()
if first_comment:
shutil.copy(TEMPLATE_DIR / "comments.xml", comments) shutil.copy(TEMPLATE_DIR / "comments.xml", comments)
_ensure_comment_relationships(Path(unpacked_dir)) _ensure_comment_relationships(unpacked_dir)
_ensure_comment_content_types(Path(unpacked_dir)) _ensure_comment_content_types(unpacked_dir)
_append_xml( _append_xml(
comments, comments,
"w:comments", "w:comments",
COMMENT_XML.format( COMMENT_XML.format(
id=comment_id, id=comment_id, author=author, date=ts, initials=initials,
author=author, para_id=para_id, text=text,
date=ts,
initials=initials,
para_id=para_id,
text=text,
), ),
) )
ext = word / "commentsExtended.xml" ext = word / "commentsExtended.xml"
if not ext.exists(): if not ext.exists():
shutil.copy(TEMPLATE_DIR / "commentsExtended.xml", ext) shutil.copy(TEMPLATE_DIR / "commentsExtended.xml", ext)
if parent_id is not None: if parent_para is not None:
parent_para = _find_para_id(comments, parent_id)
if not parent_para:
return "", f"Error: Parent comment {parent_id} not found"
_append_xml( _append_xml(
ext, ext, "w15:commentsEx",
"w15:commentsEx",
f'<w15:commentEx w15:paraId="{para_id}" w15:paraIdParent="{parent_para}" w15:done="0"/>', f'<w15:commentEx w15:paraId="{para_id}" w15:paraIdParent="{parent_para}" w15:done="0"/>',
) )
else: else:
_append_xml( _append_xml(
ext, ext, "w15:commentsEx",
"w15:commentsEx",
f'<w15:commentEx w15:paraId="{para_id}" w15:done="0"/>', f'<w15:commentEx w15:paraId="{para_id}" w15:done="0"/>',
) )
@@ -272,8 +294,7 @@ def add_comment(
if not ids.exists(): if not ids.exists():
shutil.copy(TEMPLATE_DIR / "commentsIds.xml", ids) shutil.copy(TEMPLATE_DIR / "commentsIds.xml", ids)
_append_xml( _append_xml(
ids, ids, "w16cid:commentsIds",
"w16cid:commentsIds",
f'<w16cid:commentId w16cid:paraId="{para_id}" w16cid:durableId="{durable_id}"/>', f'<w16cid:commentId w16cid:paraId="{para_id}" w16cid:durableId="{durable_id}"/>',
) )
@@ -281,38 +302,67 @@ def add_comment(
if not extensible.exists(): if not extensible.exists():
shutil.copy(TEMPLATE_DIR / "commentsExtensible.xml", extensible) shutil.copy(TEMPLATE_DIR / "commentsExtensible.xml", extensible)
_append_xml( _append_xml(
extensible, extensible, "w16cex:commentsExtensible",
"w16cex:commentsExtensible",
f'<w16cex:commentExtensible w16cex:durableId="{durable_id}" w16cex:dateUtc="{ts}"/>', f'<w16cex:commentExtensible w16cex:durableId="{durable_id}" w16cex:dateUtc="{ts}"/>',
) )
action = "reply" if parent_id is not None else "comment" action = "reply" if parent_id is not None else "comment"
return para_id, f"Added {action} {comment_id} (para_id={para_id})" return comment_id, para_id, f"Added {action} id={comment_id} (paraId={para_id})"
if __name__ == "__main__": def main() -> None:
p = argparse.ArgumentParser(description="Add comments to DOCX documents") p = argparse.ArgumentParser(description="Add a comment to a DOCX (directory or .docx file).")
p.add_argument("unpacked_dir", help="Unpacked DOCX directory") p.add_argument("input", help="Unpacked DOCX directory OR a .docx/.dotx file")
p.add_argument("comment_id", type=int, help="Comment ID (must be unique)") p.add_argument("text", help="Comment text (plain text; XML-escaped automatically)")
p.add_argument("text", help="Comment text") p.add_argument("--raw", action="store_true",
help="Treat text as pre-escaped XML (skip automatic escaping)")
p.add_argument("--id", type=int, dest="comment_id",
help="Comment ID (default: auto-assign as max existing + 1)")
p.add_argument("--author", default="Claude", help="Author name") p.add_argument("--author", default="Claude", help="Author name")
p.add_argument("--initials", default="C", help="Author initials") p.add_argument("--initials", default="C", help="Author initials")
p.add_argument("--parent", type=int, help="Parent comment ID (for replies)") p.add_argument("--parent", type=int, help="Parent comment ID (makes this a reply)")
p.add_argument("-o", "--output",
help="Output .docx path (only used when input is a .docx; default: overwrite input)")
args = p.parse_args() args = p.parse_args()
para_id, msg = add_comment( src = Path(args.input)
args.unpacked_dir,
args.comment_id, try:
args.text, if src.is_dir():
args.author, if args.output:
args.initials, print("Warning: --output ignored for directory input", file=sys.stderr)
args.parent, cid, _, msg = add_comment(
) src, args.text, comment_id=args.comment_id,
print(msg) author=args.author, initials=args.initials,
if "Error" in msg: parent_id=args.parent, raw=args.raw,
)
print(msg)
elif src.is_file() and src.suffix.lower() in (".docx", ".dotx"):
out = Path(args.output) if args.output else src
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
with zipfile.ZipFile(src) as zf:
_safe_extract(zf, tmp_path)
cid, _, msg = add_comment(
tmp_path, args.text, comment_id=args.comment_id,
author=args.author, initials=args.initials,
parent_id=args.parent, raw=args.raw,
)
_rezip(tmp_path, out)
print(msg)
print(f"Wrote {out} (comment defined; add markers to word/document.xml to make it visible)")
else:
print(f"Error: {src} is neither a directory nor a .docx/.dotx file", file=sys.stderr)
sys.exit(1)
except (FileNotFoundError, ValueError, zipfile.BadZipFile, ExpatError) as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1) sys.exit(1)
cid = args.comment_id
if args.parent is not None: if args.parent is not None:
print(REPLY_MARKER_TEMPLATE.format(pid=args.parent, cid=cid)) print(REPLY_MARKER_TEMPLATE.format(pid=args.parent, cid=cid))
else: else:
print(COMMENT_MARKER_TEMPLATE.format(cid=cid)) print(COMMENT_MARKER_TEMPLATE.format(cid=cid))
if __name__ == "__main__":
main()
skills/docx/scripts/merge_runs.py
Executable
+310
View File
@@ -0,0 +1,310 @@
"""Merge adjacent identically-formatted runs in a DOCX.
Word fragments paragraph text across many <w:r> elements (revision ids,
spell-check markers, editing history), which makes find-and-replace on
word/document.xml unreliable — the string you're looking for is split
across runs. This coalesces adjacent runs whose formatting (<w:rPr>) is
identical, strips rsid attributes and proofErr markers, and consolidates the
text elements — <w:t>, and <w:delText> for text inside a tracked deletion.
Rendering is unchanged. The text you search is what Word draws, which is not
always the bytes in the file: an element without xml:space="preserve" has its
edge whitespace trimmed before it reaches the page, so `<w:t>Hello </w:t>`
followed by `<w:t>world</w:t>` reads "Helloworld" and merges to exactly that.
Runs in two different <w:ins>/<w:del> wrappers are never merged: that would
rewrite tracked-change structure, collapsing separate revisions into one.
Only word/document.xml is processed (not headers, footers, or footnotes).
Usage:
python merge_runs.py unpacked/ # after unzip, before editing
python merge_runs.py document.docx # rewrite in place
python merge_runs.py document.docx -o out.docx
"""
import argparse
import sys
import tempfile
import zipfile
from pathlib import Path
import defusedxml.minidom
from office.helpers import XML_SPACE, rendered_text, rezip, safe_extract
WORDML_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
def merge_runs(input_dir: str) -> tuple[int, str]:
doc_xml = Path(input_dir) / "word" / "document.xml"
if not doc_xml.exists():
return 0, f"Error: {doc_xml} not found"
try:
dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8"))
root = dom.documentElement
run_names = _run_tag_names(root)
_remove_elements(root, "proofErr")
runs = _find_runs(root, run_names)
_strip_rsid_attrs(runs)
merge_count = 0
for container in {run.parentNode for run in runs}:
merge_count += _merge_runs_in(container, run_names)
doc_xml.write_bytes(dom.toxml(encoding="UTF-8"))
return merge_count, f"Merged {merge_count} runs"
except Exception as e:
return 0, f"Error: {e}"
def _is_element(node, tag: str) -> bool:
name = node.localName or node.tagName
return name == tag or name.endswith(f":{tag}")
def _run_tag_names(root) -> set[str]:
names = set()
for attr in root.attributes.values():
if attr.value == WORDML_NS:
if attr.name == "xmlns":
names.add("r")
elif attr.name.startswith("xmlns:"):
names.add(attr.name.split(":", 1)[1] + ":r")
return names or {"w:r", "r"}
def _find_elements(root, tag: str) -> list:
results = []
def traverse(node):
if node.nodeType == node.ELEMENT_NODE:
if _is_element(node, tag):
results.append(node)
for child in node.childNodes:
traverse(child)
traverse(root)
return results
def _find_runs(root, run_names: set[str]) -> list:
return [e for e in _find_elements(root, "r") if _is_run(e, run_names)]
def _get_child(parent, tag: str):
return next(iter(_get_children(parent, tag)), None)
def _get_children(parent, tag: str) -> list:
return [
child
for child in parent.childNodes
if child.nodeType == child.ELEMENT_NODE and _is_element(child, tag)
]
def _is_adjacent(elem1, elem2) -> bool:
node = elem1.nextSibling
while node:
if node == elem2:
return True
if node.nodeType == node.ELEMENT_NODE:
return False
if node.nodeType == node.TEXT_NODE and node.data.strip(XML_SPACE):
return False
node = node.nextSibling
return False
def _remove_elements(root, tag: str):
for elem in _find_elements(root, tag):
if elem.parentNode:
elem.parentNode.removeChild(elem)
def _strip_rsid_attrs(runs: list):
for run in runs:
for attr in list(run.attributes.values()):
if "rsid" in attr.name.lower():
run.removeAttribute(attr.name)
def _merge_runs_in(container, run_names: set[str]) -> int:
merge_count = 0
run = _first_child_run(container, run_names)
while run:
while True:
next_elem = _next_element_sibling(run)
if next_elem and _is_run(next_elem, run_names) and _can_merge(run, next_elem):
_merge_run_content(run, next_elem)
container.removeChild(next_elem)
merge_count += 1
else:
break
_consolidate_text(run)
run = _next_sibling_run(run, run_names)
return merge_count
def _first_child_run(container, run_names: set[str]):
for child in container.childNodes:
if child.nodeType == child.ELEMENT_NODE and _is_run(child, run_names):
return child
return None
def _next_element_sibling(node):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
return sibling
sibling = sibling.nextSibling
return None
def _next_sibling_run(node, run_names: set[str]):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
if _is_run(sibling, run_names):
return sibling
sibling = sibling.nextSibling
return None
def _is_run(node, run_names: set[str]) -> bool:
return node.tagName in run_names
def _can_merge(run1, run2) -> bool:
rpr1 = _get_child(run1, "rPr")
rpr2 = _get_child(run2, "rPr")
if (rpr1 is None) != (rpr2 is None):
return False
if rpr1 is None:
return True
return rpr1.toxml() == rpr2.toxml()
def _merge_run_content(target, source):
for child in list(source.childNodes):
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name != "rPr" and not name.endswith(":rPr"):
target.appendChild(child)
def _element_text(elem) -> str:
return "".join(
child.data
for child in elem.childNodes
if child.nodeType in (child.TEXT_NODE, child.CDATA_SECTION_NODE)
)
def _has_preserve(elem) -> bool:
return elem.getAttribute("xml:space") == "preserve"
def _rendered_text(elem) -> str:
return rendered_text(_element_text(elem), _has_preserve(elem))
def _consolidate_text(run):
for tag in ("t", "delText"):
_consolidate_text_elements(run, tag)
def _consolidate_text_elements(run, tag: str):
t_elements = _get_children(run, tag)
for i in range(len(t_elements) - 1, 0, -1):
curr, prev = t_elements[i], t_elements[i - 1]
if _is_adjacent(prev, curr):
merged = _rendered_text(prev) + _rendered_text(curr)
had_preserve = _has_preserve(prev) or _has_preserve(curr)
new_text = run.ownerDocument.createTextNode(merged)
for node in list(prev.childNodes):
if node.nodeType in (node.TEXT_NODE, node.CDATA_SECTION_NODE):
prev.removeChild(node)
else:
run.insertBefore(node, curr)
prev.appendChild(new_text)
for node in list(curr.childNodes):
if node.nodeType not in (node.TEXT_NODE, node.CDATA_SECTION_NODE):
run.insertBefore(node, curr)
if merged != merged.strip(XML_SPACE) or had_preserve:
prev.setAttribute("xml:space", "preserve")
elif prev.hasAttribute("xml:space"):
prev.removeAttribute("xml:space")
run.removeChild(curr)
def _merge_or_die(path: Path) -> str:
_, msg = merge_runs(str(path))
if msg.startswith("Error"):
print(msg, file=sys.stderr)
sys.exit(1)
return msg
def main() -> None:
p = argparse.ArgumentParser(
description="Merge adjacent identically-formatted runs in a DOCX (directory or .docx file)."
)
p.add_argument("input", help="Unpacked DOCX directory OR a .docx/.dotx file")
p.add_argument(
"-o", "--output",
help="Output .docx path (only valid when input is a .docx; default: overwrite input)",
)
args = p.parse_args()
src = Path(args.input)
try:
if src.is_dir():
if args.output:
p.error("--output is only valid for .docx input; directory input is modified in place")
print(_merge_or_die(src))
elif src.is_file() and src.suffix.lower() in (".docx", ".dotx"):
out = Path(args.output) if args.output else src
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
with zipfile.ZipFile(src) as zf:
safe_extract(zf, tmp_path)
msg = _merge_or_die(tmp_path)
rezip(tmp_path, out)
print(f"{msg}; wrote {out}")
else:
print(f"Error: {src} is neither a directory nor a .docx/.dotx file", file=sys.stderr)
sys.exit(1)
except (OSError, ValueError, zipfile.BadZipFile) as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
skills/docx/scripts/office/helpers/__init__.py
+111
View File
@@ -0,0 +1,111 @@
import os
import posixpath
import re
import stat
import tempfile
import urllib.parse
import zipfile
from pathlib import Path
OOXML_FAMILY = {
".docx": "docx",
".dotx": "docx",
".pptx": "pptx",
".potx": "pptx",
".xlsx": "xlsx",
".xltx": "xlsx",
}
_SCHEME_RE = re.compile(r"^[A-Za-z][A-Za-z0-9+.\-]*:")
SLIDE_REL_TYPE = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/slide"
def opc_target(target: str, source_part: str, target_mode: str = "") -> str | None:
if not target:
return None
if target_mode.lower() == "external":
return None
if _SCHEME_RE.match(target):
return None
target = urllib.parse.unquote(target)
if "\\" in target:
raise ValueError(f"relationship target is not a POSIX part name: {target!r}")
if target.startswith("/"):
joined = target.lstrip("/")
else:
joined = posixpath.join(posixpath.dirname(source_part), target)
parts: list[str] = []
for segment in posixpath.normpath(joined).split("/"):
if segment in ("", "."):
continue
if segment == "..":
if not parts:
raise ValueError(f"relationship target escapes the package: {target!r}")
parts.pop()
else:
parts.append(segment)
if not parts:
raise ValueError(f"relationship target resolves to nothing: {target!r}")
return "/".join(parts)
def rels_source_part(rels_file: Path, unpacked_dir: Path) -> str:
owner_dir = rels_file.parent.parent.relative_to(unpacked_dir)
return posixpath.join(owner_dir.as_posix(), rels_file.name[: -len(".rels")]).lstrip("./")
def part_text(data: bytes) -> str:
return data.decode("utf-8", "surrogateescape")
XML_SPACE = " \t\r\n"
def rendered_text(text: str, preserve: bool) -> str:
return text if preserve else text.strip(XML_SPACE)
def safe_extract(zf: zipfile.ZipFile, dest: Path) -> None:
dest = dest.resolve()
for m in zf.infolist():
if stat.S_ISLNK(m.external_attr >> 16):
raise ValueError(f"symlink archive entry not allowed: {m.filename!r}")
target = (dest / m.filename).resolve()
if not target.is_relative_to(dest):
raise ValueError(f"unsafe archive entry: {m.filename!r}")
zf.extract(m, dest)
def rezip(src_dir: Path, out_path: Path) -> None:
files = sorted(p for p in src_dir.rglob("*") if p.is_file())
ct = src_dir / "[Content_Types].xml"
fd, tmp_name = tempfile.mkstemp(
prefix=out_path.name + ".", suffix=".tmp", dir=out_path.parent
)
tmp_out = Path(tmp_name)
try:
with os.fdopen(fd, "wb") as fh:
with zipfile.ZipFile(fh, "w", zipfile.ZIP_DEFLATED) as zf:
if ct.exists():
zf.write(ct, ct.relative_to(src_dir), compress_type=zipfile.ZIP_STORED)
for f in files:
if f == ct:
continue
zf.write(f, f.relative_to(src_dir))
if out_path.exists():
mode = out_path.stat().st_mode & 0o777
else:
umask = os.umask(0)
os.umask(umask)
mode = 0o666 & ~umask
os.chmod(tmp_out, mode)
os.replace(tmp_out, out_path)
finally:
if tmp_out.exists():
tmp_out.unlink()
skills/docx/scripts/office/helpers/merge_runs.py
-199
View File
@@ -1,199 +0,0 @@
"""Merge adjacent runs with identical formatting in DOCX.
Merges adjacent <w:r> elements that have identical <w:rPr> properties.
Works on runs in paragraphs and inside tracked changes (<w:ins>, <w:del>).
Also:
- Removes rsid attributes from runs (revision metadata that doesn't affect rendering)
- Removes proofErr elements (spell/grammar markers that block merging)
"""
from pathlib import Path
import defusedxml.minidom
def merge_runs(input_dir: str) -> tuple[int, str]:
doc_xml = Path(input_dir) / "word" / "document.xml"
if not doc_xml.exists():
return 0, f"Error: {doc_xml} not found"
try:
dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8"))
root = dom.documentElement
_remove_elements(root, "proofErr")
_strip_run_rsid_attrs(root)
containers = {run.parentNode for run in _find_elements(root, "r")}
merge_count = 0
for container in containers:
merge_count += _merge_runs_in(container)
doc_xml.write_bytes(dom.toxml(encoding="UTF-8"))
return merge_count, f"Merged {merge_count} runs"
except Exception as e:
return 0, f"Error: {e}"
def _find_elements(root, tag: str) -> list:
results = []
def traverse(node):
if node.nodeType == node.ELEMENT_NODE:
name = node.localName or node.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(node)
for child in node.childNodes:
traverse(child)
traverse(root)
return results
def _get_child(parent, tag: str):
for child in parent.childNodes:
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name == tag or name.endswith(f":{tag}"):
return child
return None
def _get_children(parent, tag: str) -> list:
results = []
for child in parent.childNodes:
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(child)
return results
def _is_adjacent(elem1, elem2) -> bool:
node = elem1.nextSibling
while node:
if node == elem2:
return True
if node.nodeType == node.ELEMENT_NODE:
return False
if node.nodeType == node.TEXT_NODE and node.data.strip():
return False
node = node.nextSibling
return False
def _remove_elements(root, tag: str):
for elem in _find_elements(root, tag):
if elem.parentNode:
elem.parentNode.removeChild(elem)
def _strip_run_rsid_attrs(root):
for run in _find_elements(root, "r"):
for attr in list(run.attributes.values()):
if "rsid" in attr.name.lower():
run.removeAttribute(attr.name)
def _merge_runs_in(container) -> int:
merge_count = 0
run = _first_child_run(container)
while run:
while True:
next_elem = _next_element_sibling(run)
if next_elem and _is_run(next_elem) and _can_merge(run, next_elem):
_merge_run_content(run, next_elem)
container.removeChild(next_elem)
merge_count += 1
else:
break
_consolidate_text(run)
run = _next_sibling_run(run)
return merge_count
def _first_child_run(container):
for child in container.childNodes:
if child.nodeType == child.ELEMENT_NODE and _is_run(child):
return child
return None
def _next_element_sibling(node):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
return sibling
sibling = sibling.nextSibling
return None
def _next_sibling_run(node):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
if _is_run(sibling):
return sibling
sibling = sibling.nextSibling
return None
def _is_run(node) -> bool:
name = node.localName or node.tagName
return name == "r" or name.endswith(":r")
def _can_merge(run1, run2) -> bool:
rpr1 = _get_child(run1, "rPr")
rpr2 = _get_child(run2, "rPr")
if (rpr1 is None) != (rpr2 is None):
return False
if rpr1 is None:
return True
return rpr1.toxml() == rpr2.toxml()
def _merge_run_content(target, source):
for child in list(source.childNodes):
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name != "rPr" and not name.endswith(":rPr"):
target.appendChild(child)
def _consolidate_text(run):
t_elements = _get_children(run, "t")
for i in range(len(t_elements) - 1, 0, -1):
curr, prev = t_elements[i], t_elements[i - 1]
if _is_adjacent(prev, curr):
prev_text = prev.firstChild.data if prev.firstChild else ""
curr_text = curr.firstChild.data if curr.firstChild else ""
merged = prev_text + curr_text
if prev.firstChild:
prev.firstChild.data = merged
else:
prev.appendChild(run.ownerDocument.createTextNode(merged))
if merged.startswith(" ") or merged.endswith(" "):
prev.setAttribute("xml:space", "preserve")
elif prev.hasAttribute("xml:space"):
prev.removeAttribute("xml:space")
run.removeChild(curr)
skills/docx/scripts/office/helpers/pptx_chart.py
+170
View File
@@ -0,0 +1,170 @@
"""Find chart XML that PowerPoint refuses but the schema accepts.
Detection only: for either fault more than one repair is valid, and only the
author knows which was meant.
"""
from __future__ import annotations
import re
from typing import Mapping
from . import part_text
_CHART_PART_RE = re.compile(r"ppt/charts/chart\d+\.xml")
_GROUPING_RE = re.compile(r"""<c:grouping\b[^>]*?\bval=["'](\w+)["']""")
_DLBL_POS_RE = re.compile(r"""<c:dLblPos\b[^>]*?\bval=["'](\w+)["']""")
def _strip_ext_lst(text: str) -> str:
out, cursor = [], 0
for lo, hi in _ext_lst_spans(text):
out.append(text[cursor:lo])
cursor = hi
out.append(text[cursor:])
return "".join(out)
_BAR_GROUP_RE = re.compile(r"<c:(bar3DChart|barChart)\b[^>]*(?<!/)>.*?</c:\1\s*>", re.DOTALL)
STACKED_GROUPINGS = frozenset({"stacked", "percentStacked"})
ILLEGAL_ON_STACKED = frozenset({"outEnd"})
LEGAL_ON_STACKED = ("ctr", "inEnd", "inBase")
def _check_stacked_label_positions(part: str, xml: str) -> list[str]:
problems: list[str] = []
for match in _BAR_GROUP_RE.finditer(xml):
block = _strip_ext_lst(match.group(0))
group = match.group(1)
grouping = _GROUPING_RE.search(block)
if grouping is None or grouping.group(1) not in STACKED_GROUPINGS:
continue
bad = [p for p in _DLBL_POS_RE.findall(block) if p in ILLEGAL_ON_STACKED]
for pos in sorted(set(bad)):
problems.append(
f'{part}: {bad.count(pos)} data label(s) use dLblPos="{pos}" on a '
f"{grouping.group(1)} {group}; PowerPoint allows only "
f"{', '.join(LEGAL_ON_STACKED)} there"
)
return problems
_ANY_CHART_GROUP_RE = re.compile(r"<c:(\w+Chart)\b[^>]*(?<!/)>.*?</c:\1\s*>", re.DOTALL)
_AXID_RE = re.compile(
r"""\s*<c:axId\b[^>]*?\bval=["'](-?\d+)["']\s*(?:/>|>\s*</c:axId\s*>)"""
)
_AXIS_DECL_RE = re.compile(
r"""<c:(catAx|valAx|serAx|dateAx)\b[^>]*(?<!/)>\s*<c:axId\b[^>]*?\bval=["'](-?\d+)["']"""
)
AXID_LIMIT = {
"barChart": 2, "lineChart": 2, "areaChart": 2, "scatterChart": 2,
"bubbleChart": 2, "radarChart": 2, "stockChart": 2,
"bar3DChart": 3, "line3DChart": 3, "area3DChart": 3,
"surfaceChart": 3, "surface3DChart": 3,
}
AXID_MINIMUM = {
"barChart": 2, "lineChart": 2, "areaChart": 2, "scatterChart": 2,
"bubbleChart": 2, "radarChart": 2, "stockChart": 2,
"bar3DChart": 2, "area3DChart": 2, "surfaceChart": 2,
"line3DChart": 3, "surface3DChart": 3,
}
def _declared_axes(xml: str) -> dict[str, list[str]]:
axes: dict[str, list[str]] = {}
for kind, axid in _AXIS_DECL_RE.findall(xml):
axes.setdefault(kind, []).append(axid)
return axes
def _canonical_ids(axes: dict[str, list[str]], limit: int) -> list[str] | None:
category = axes.get("catAx", []) + axes.get("dateAx", [])
value = axes.get("valAx", [])
series = axes.get("serAx", [])
if len(category) != 1 or len(value) != 1 or len(series) > 1:
return None
ids = [category[0], value[0]]
if limit >= 3 and series:
ids.append(series[0])
return ids
def _undeclared_axes(kind: str, block: str, axes: dict[str, list[str]]) -> list[str] | None:
if kind not in AXID_LIMIT:
return None
ids = _AXID_RE.findall(block)
declared = {i for group in axes.values() for i in group}
if len([i for i in ids if i in declared]) >= 2:
return None
return ids
def _check_chart_axis_references(part: str, xml: str) -> list[str]:
axes = _declared_axes(xml)
problems: list[str] = []
declared = {i for group in axes.values() for i in group}
for match in _ANY_CHART_GROUP_RE.finditer(xml):
kind, block = match.group(1), match.group(0)
ids = _undeclared_axes(kind, block, axes)
if ids is None:
continue
if not ids:
problems.append(
f"{part}: <c:{kind}> declares no <c:axId> this part can resolve; a chart "
f"group needs {AXID_MINIMUM[kind]}, and PowerPoint discards one with fewer"
)
continue
dead = [i for i in ids if i not in declared]
canonical = _canonical_ids(axes, AXID_LIMIT[kind])
if canonical is not None and len(canonical) >= AXID_MINIMUM[kind]:
hint = f"Fix: point them at the axes this part declares ({', '.join(canonical)})"
else:
hint = ("Fix: the part declares several axes of a kind -- declare the "
"secondary axes the series expects, or drop them")
detail = (f"of which {', '.join(dead)} name no declared axis"
if dead else f"only {len(ids)} of which this part declares")
problems.append(
f"{part}: <c:{kind}> references axId {', '.join(ids)}, {detail}, "
f"leaving fewer than two live axes; PowerPoint discards the chart. {hint}"
)
return problems
def _ext_lst_spans(text: str) -> list[tuple[int, int]]:
spans: list[tuple[int, int]] = []
depth = 0
start = 0
for match in re.finditer(r"<(/?)c:extLst\b[^>]*?(/?)>", text):
closing, self_closing = match.group(1), match.group(2)
if self_closing:
continue
if closing:
depth -= 1
if depth == 0:
spans.append((start, match.end()))
else:
if depth == 0:
start = match.start()
depth += 1
return spans
CHART_CHECKS = (_check_stacked_label_positions, _check_chart_axis_references)
def find_chart_problems(files: Mapping[str, bytes]) -> list[str]:
problems: list[str] = []
for part in sorted(n for n in files if _CHART_PART_RE.fullmatch(n)):
xml = part_text(files[part])
for check in CHART_CHECKS:
problems.extend(check(part, xml))
return problems
skills/docx/scripts/office/helpers/pptx_slide.py
+60
View File
@@ -0,0 +1,60 @@
"""Pick the slide-XML schema errors PowerPoint refuses the file over.
A denylist over lxml's messages, so an unrecognised error class is a miss rather
than a false alarm.
"""
from __future__ import annotations
import re
SLIDE_PART_RE = re.compile(
r"ppt/(slides|slideLayouts|slideMasters|notesSlides|notesMasters|handoutMasters)"
r"/[^/]+\.xml"
)
FATAL_SLIDE_ERRORS: tuple[tuple[re.Pattern[str], str], ...] = (
(
re.compile(r"\}tableStyleId': This element is not expected"),
"two <a:tableStyleId> in one <a:tblPr> (the schema allows one)",
),
(
re.compile(r"\}srgbClr', attribute 'val'"),
"a colour that is not six hex digits",
),
(
re.compile(r"\}txBody': Missing child element"),
"a <p:txBody> with no children",
),
(
re.compile(r"\}miter', attribute 'lim'"),
'a line join with lim="NaN"',
),
(
re.compile(r"\}uLnTx': This element is not expected"),
"<a:uLnTx> in a position the schema forbids",
),
(
re.compile(r"\}overrideClrMapping': This element is not expected"),
"<p:overrideClrMapping> in a position the schema forbids",
),
(
re.compile(r"\}nvGrpSpPr': Missing child element"),
"a <p:nvGrpSpPr> with no children",
),
)
def is_schema_verdict(error: str) -> bool:
return error.startswith("Element ")
def fatal_slide_errors(errors: set[str]) -> list[str]:
out = []
for error in sorted(errors):
for pattern, meaning in FATAL_SLIDE_ERRORS:
if pattern.search(error):
out.append(f"{meaning}: {error}")
break
return out
skills/docx/scripts/office/helpers/pptx_theme.py
+114
View File
@@ -0,0 +1,114 @@
"""Find masters sharing a theme part in the way PowerPoint refuses to open.
Reports only; the fix is to move <p:notesMasterIdLst> back to directly after
<p:sldIdLst> in ppt/presentation.xml.
"""
from __future__ import annotations
import posixpath
import re
from typing import Mapping
from . import part_text
THEME_REL_TYPE = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/theme"
_MASTER_RE = re.compile(
r"^ppt/(?P<group>slideMasters|notesMasters|handoutMasters)/"
r"(?:slide|notes|handout)Master(?P<num>\d+)\.xml$"
)
_GROUP_ORDER = {"slideMasters": 0, "notesMasters": 1, "handoutMasters": 2}
_RELATIONSHIP_RE = re.compile(
r"<Relationship\b[^>]*?(?:/>|>.*?</Relationship\s*>)", re.DOTALL
)
def _sort_key(name: str) -> tuple[int, int]:
m = _MASTER_RE.match(name)
assert m is not None
return (_GROUP_ORDER[m.group("group")], int(m.group("num")))
def _rels_path(part: str) -> str:
directory, base = posixpath.split(part)
return f"{directory}/_rels/{base}.rels"
def _resolve(rels_path: str, target: str) -> str:
if target.startswith("/"):
return target.lstrip("/")
part_dir = posixpath.dirname(posixpath.dirname(rels_path))
return posixpath.normpath(posixpath.join(part_dir, target))
def _theme_rel(files: Mapping[str, bytes], master: str):
rels_path = _rels_path(master)
rels = files.get(rels_path)
if rels is None:
return None
for element in _RELATIONSHIP_RE.findall(part_text(rels)):
if f'Type="{THEME_REL_TYPE}"' not in element:
continue
target = re.search(r'\bTarget="([^"]+)"', element)
if target is None:
continue
return rels_path, element, _resolve(rels_path, target.group(1))
return None
def _masters(files: Mapping[str, bytes]) -> list[str]:
return sorted((n for n in files if _MASTER_RE.match(n)), key=_sort_key)
_PRESENTATION = "ppt/presentation.xml"
_NOTES_MASTERS = "ppt/notesMasters/"
_IGNORABLE_RE = re.compile(r"<!--.*?-->|<\?.*?\?>", re.DOTALL)
_AFTER_SLDIDLST_RE = re.compile(
r"<p:sldIdLst\b(?:[^>]*/>|[^>]*>.*?</p:sldIdLst\s*>)\s*(<[^>\s/]+)", re.DOTALL
)
def _notes_master_share_is_inert(files: Mapping[str, bytes]) -> bool:
data = files.get(_PRESENTATION)
if data is None:
return False
match = _AFTER_SLDIDLST_RE.search(_IGNORABLE_RE.sub("", part_text(data)))
return match is not None and match.group(1) == "<p:notesMasterIdLst"
def _shares(files: Mapping[str, bytes]):
owner: dict[str, str] = {}
for master in _masters(files):
found = _theme_rel(files, master)
if found is None:
continue
rels_path, element, theme = found
if theme not in files:
continue
if theme in owner:
yield master, rels_path, element, theme, owner[theme]
else:
owner[theme] = master
def _is_inert(master: str, inert_notes: bool) -> bool:
return inert_notes and master.startswith(_NOTES_MASTERS)
def find_shared_master_themes(files: Mapping[str, bytes]) -> list[str]:
return [
f"{master} shares {theme} with {first}"
for master, _, _, theme, first in _shares(files)
]
def live_shared_master_themes(files: Mapping[str, bytes]) -> list[str]:
inert_notes = _notes_master_share_is_inert(files)
return [
f"{master} shares {theme} with {first}"
for master, _, _, theme, first in _shares(files)
if not _is_inert(master, inert_notes)
]
skills/docx/scripts/office/helpers/simplify_redlines.py
-197
View File
@@ -1,197 +0,0 @@
"""Simplify tracked changes by merging adjacent w:ins or w:del elements.
Merges adjacent <w:ins> elements from the same author into a single element.
Same for <w:del> elements. This makes heavily-redlined documents easier to
work with by reducing the number of tracked change wrappers.
Rules:
- Only merges w:ins with w:ins, w:del with w:del (same element type)
- Only merges if same author (ignores timestamp differences)
- Only merges if truly adjacent (only whitespace between them)
"""
import xml.etree.ElementTree as ET
import zipfile
from pathlib import Path
import defusedxml.minidom
WORD_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
def simplify_redlines(input_dir: str) -> tuple[int, str]:
doc_xml = Path(input_dir) / "word" / "document.xml"
if not doc_xml.exists():
return 0, f"Error: {doc_xml} not found"
try:
dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8"))
root = dom.documentElement
merge_count = 0
containers = _find_elements(root, "p") + _find_elements(root, "tc")
for container in containers:
merge_count += _merge_tracked_changes_in(container, "ins")
merge_count += _merge_tracked_changes_in(container, "del")
doc_xml.write_bytes(dom.toxml(encoding="UTF-8"))
return merge_count, f"Simplified {merge_count} tracked changes"
except Exception as e:
return 0, f"Error: {e}"
def _merge_tracked_changes_in(container, tag: str) -> int:
merge_count = 0
tracked = [
child
for child in container.childNodes
if child.nodeType == child.ELEMENT_NODE and _is_element(child, tag)
]
if len(tracked) < 2:
return 0
i = 0
while i < len(tracked) - 1:
curr = tracked[i]
next_elem = tracked[i + 1]
if _can_merge_tracked(curr, next_elem):
_merge_tracked_content(curr, next_elem)
container.removeChild(next_elem)
tracked.pop(i + 1)
merge_count += 1
else:
i += 1
return merge_count
def _is_element(node, tag: str) -> bool:
name = node.localName or node.tagName
return name == tag or name.endswith(f":{tag}")
def _get_author(elem) -> str:
author = elem.getAttribute("w:author")
if not author:
for attr in elem.attributes.values():
if attr.localName == "author" or attr.name.endswith(":author"):
return attr.value
return author
def _can_merge_tracked(elem1, elem2) -> bool:
if _get_author(elem1) != _get_author(elem2):
return False
node = elem1.nextSibling
while node and node != elem2:
if node.nodeType == node.ELEMENT_NODE:
return False
if node.nodeType == node.TEXT_NODE and node.data.strip():
return False
node = node.nextSibling
return True
def _merge_tracked_content(target, source):
while source.firstChild:
child = source.firstChild
source.removeChild(child)
target.appendChild(child)
def _find_elements(root, tag: str) -> list:
results = []
def traverse(node):
if node.nodeType == node.ELEMENT_NODE:
name = node.localName or node.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(node)
for child in node.childNodes:
traverse(child)
traverse(root)
return results
def get_tracked_change_authors(doc_xml_path: Path) -> dict[str, int]:
if not doc_xml_path.exists():
return {}
try:
tree = ET.parse(doc_xml_path)
root = tree.getroot()
except ET.ParseError:
return {}
namespaces = {"w": WORD_NS}
author_attr = f"{{{WORD_NS}}}author"
authors: dict[str, int] = {}
for tag in ["ins", "del"]:
for elem in root.findall(f".//w:{tag}", namespaces):
author = elem.get(author_attr)
if author:
authors[author] = authors.get(author, 0) + 1
return authors
def _get_authors_from_docx(docx_path: Path) -> dict[str, int]:
try:
with zipfile.ZipFile(docx_path, "r") as zf:
if "word/document.xml" not in zf.namelist():
return {}
with zf.open("word/document.xml") as f:
tree = ET.parse(f)
root = tree.getroot()
namespaces = {"w": WORD_NS}
author_attr = f"{{{WORD_NS}}}author"
authors: dict[str, int] = {}
for tag in ["ins", "del"]:
for elem in root.findall(f".//w:{tag}", namespaces):
author = elem.get(author_attr)
if author:
authors[author] = authors.get(author, 0) + 1
return authors
except (zipfile.BadZipFile, ET.ParseError):
return {}
def infer_author(modified_dir: Path, original_docx: Path, default: str = "Claude") -> str:
modified_xml = modified_dir / "word" / "document.xml"
modified_authors = get_tracked_change_authors(modified_xml)
if not modified_authors:
return default
original_authors = _get_authors_from_docx(original_docx)
new_changes: dict[str, int] = {}
for author, count in modified_authors.items():
original_count = original_authors.get(author, 0)
diff = count - original_count
if diff > 0:
new_changes[author] = diff
if not new_changes:
return default
if len(new_changes) == 1:
return next(iter(new_changes))
raise ValueError(
f"Multiple authors added new changes: {new_changes}. "
"Cannot infer which author to validate."
)
skills/docx/scripts/office/pack.py
-159
View File
@@ -1,159 +0,0 @@
"""Pack a directory into a DOCX, PPTX, or XLSX file.
Validates with auto-repair, condenses XML formatting, and creates the Office file.
Usage:
python pack.py <input_directory> <output_file> [--original <file>] [--validate true|false]
Examples:
python pack.py unpacked/ output.docx --original input.docx
python pack.py unpacked/ output.pptx --validate false
"""
import argparse
import sys
import shutil
import tempfile
import zipfile
from pathlib import Path
import defusedxml.minidom
from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator
def pack(
input_directory: str,
output_file: str,
original_file: str | None = None,
validate: bool = True,
infer_author_func=None,
) -> tuple[None, str]:
input_dir = Path(input_directory)
output_path = Path(output_file)
suffix = output_path.suffix.lower()
if not input_dir.is_dir():
return None, f"Error: {input_dir} is not a directory"
if suffix not in {".docx", ".pptx", ".xlsx"}:
return None, f"Error: {output_file} must be a .docx, .pptx, or .xlsx file"
if validate and original_file:
original_path = Path(original_file)
if original_path.exists():
success, output = _run_validation(
input_dir, original_path, suffix, infer_author_func
)
if output:
print(output)
if not success:
return None, f"Error: Validation failed for {input_dir}"
with tempfile.TemporaryDirectory() as temp_dir:
temp_content_dir = Path(temp_dir) / "content"
shutil.copytree(input_dir, temp_content_dir)
for pattern in ["*.xml", "*.rels"]:
for xml_file in temp_content_dir.rglob(pattern):
_condense_xml(xml_file)
output_path.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zf:
for f in temp_content_dir.rglob("*"):
if f.is_file():
zf.write(f, f.relative_to(temp_content_dir))
return None, f"Successfully packed {input_dir} to {output_file}"
def _run_validation(
unpacked_dir: Path,
original_file: Path,
suffix: str,
infer_author_func=None,
) -> tuple[bool, str | None]:
output_lines = []
validators = []
if suffix == ".docx":
author = "Claude"
if infer_author_func:
try:
author = infer_author_func(unpacked_dir, original_file)
except ValueError as e:
print(f"Warning: {e} Using default author 'Claude'.", file=sys.stderr)
validators = [
DOCXSchemaValidator(unpacked_dir, original_file),
RedliningValidator(unpacked_dir, original_file, author=author),
]
elif suffix == ".pptx":
validators = [PPTXSchemaValidator(unpacked_dir, original_file)]
if not validators:
return True, None
total_repairs = sum(v.repair() for v in validators)
if total_repairs:
output_lines.append(f"Auto-repaired {total_repairs} issue(s)")
success = all(v.validate() for v in validators)
if success:
output_lines.append("All validations PASSED!")
return success, "\n".join(output_lines) if output_lines else None
def _condense_xml(xml_file: Path) -> None:
try:
with open(xml_file, encoding="utf-8") as f:
dom = defusedxml.minidom.parse(f)
for element in dom.getElementsByTagName("*"):
if element.tagName.endswith(":t"):
continue
for child in list(element.childNodes):
if (
child.nodeType == child.TEXT_NODE
and child.nodeValue
and child.nodeValue.strip() == ""
) or child.nodeType == child.COMMENT_NODE:
element.removeChild(child)
xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
except Exception as e:
print(f"ERROR: Failed to parse {xml_file.name}: {e}", file=sys.stderr)
raise
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Pack a directory into a DOCX, PPTX, or XLSX file"
)
parser.add_argument("input_directory", help="Unpacked Office document directory")
parser.add_argument("output_file", help="Output Office file (.docx/.pptx/.xlsx)")
parser.add_argument(
"--original",
help="Original file for validation comparison",
)
parser.add_argument(
"--validate",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Run validation with auto-repair (default: true)",
)
args = parser.parse_args()
_, message = pack(
args.input_directory,
args.output_file,
original_file=args.original,
validate=args.validate,
)
print(message)
if "Error" in message:
sys.exit(1)
skills/docx/scripts/office/soffice.py
+17 -8
View File
@@ -4,20 +4,23 @@ sockets may be blocked (e.g., sandboxed VMs). Detects the restriction
at runtime and applies an LD_PRELOAD shim if needed. at runtime and applies an LD_PRELOAD shim if needed.
Usage: Usage:
from office.soffice import run_soffice, get_soffice_env from office.soffice import run_soffice
# Option 1 – run soffice directly
result = run_soffice(["--headless", "--convert-to", "pdf", "input.docx"]) result = run_soffice(["--headless", "--convert-to", "pdf", "input.docx"])
# Option 2 – get env dict for your own subprocess calls Call soffice through run_soffice, not through subprocess with get_soffice_env():
env = get_soffice_env() the env dict carries the shim but names no user profile, and a non-root sandbox
subprocess.run(["soffice", ...], env=env) cannot bootstrap the default one -- soffice aborts with "User installation could
not be completed" and converts nothing. get_soffice_env() stays public for the
callers that build their own argv (they must pass -env:UserInstallation too).
""" """
import contextlib
import os import os
import socket import socket
import subprocess import subprocess
import tempfile import tempfile
from collections.abc import Iterable
from pathlib import Path from pathlib import Path
@@ -32,9 +35,15 @@ def get_soffice_env() -> dict:
return env return env
def run_soffice(args: list[str], **kwargs) -> subprocess.CompletedProcess: def run_soffice(args: Iterable[str], **kwargs) -> subprocess.CompletedProcess:
env = get_soffice_env() args = list(args)
return subprocess.run(["soffice"] + args, env=env, **kwargs) with contextlib.ExitStack() as stack:
if not any(str(a).startswith("-env:UserInstallation") for a in args):
profile = stack.enter_context(
tempfile.TemporaryDirectory(prefix="lo_profile_", ignore_cleanup_errors=True)
)
args = [f"-env:UserInstallation={Path(profile).as_uri()}"] + args
return subprocess.run(["soffice"] + args, env=get_soffice_env(), **kwargs)
skills/docx/scripts/office/unpack.py
-132
View File
@@ -1,132 +0,0 @@
"""Unpack Office files (DOCX, PPTX, XLSX) for editing.
Extracts the ZIP archive, pretty-prints XML files, and optionally:
- Merges adjacent runs with identical formatting (DOCX only)
- Simplifies adjacent tracked changes from same author (DOCX only)
Usage:
python unpack.py <office_file> <output_dir> [options]
Examples:
python unpack.py document.docx unpacked/
python unpack.py presentation.pptx unpacked/
python unpack.py document.docx unpacked/ --merge-runs false
"""
import argparse
import sys
import zipfile
from pathlib import Path
import defusedxml.minidom
from helpers.merge_runs import merge_runs as do_merge_runs
from helpers.simplify_redlines import simplify_redlines as do_simplify_redlines
SMART_QUOTE_REPLACEMENTS = {
"\u201c": "&#x201C;",
"\u201d": "&#x201D;",
"\u2018": "&#x2018;",
"\u2019": "&#x2019;",
}
def unpack(
input_file: str,
output_directory: str,
merge_runs: bool = True,
simplify_redlines: bool = True,
) -> tuple[None, str]:
input_path = Path(input_file)
output_path = Path(output_directory)
suffix = input_path.suffix.lower()
if not input_path.exists():
return None, f"Error: {input_file} does not exist"
if suffix not in {".docx", ".pptx", ".xlsx"}:
return None, f"Error: {input_file} must be a .docx, .pptx, or .xlsx file"
try:
output_path.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(input_path, "r") as zf:
zf.extractall(output_path)
xml_files = list(output_path.rglob("*.xml")) + list(output_path.rglob("*.rels"))
for xml_file in xml_files:
_pretty_print_xml(xml_file)
message = f"Unpacked {input_file} ({len(xml_files)} XML files)"
if suffix == ".docx":
if simplify_redlines:
simplify_count, _ = do_simplify_redlines(str(output_path))
message += f", simplified {simplify_count} tracked changes"
if merge_runs:
merge_count, _ = do_merge_runs(str(output_path))
message += f", merged {merge_count} runs"
for xml_file in xml_files:
_escape_smart_quotes(xml_file)
return None, message
except zipfile.BadZipFile:
return None, f"Error: {input_file} is not a valid Office file"
except Exception as e:
return None, f"Error unpacking: {e}"
def _pretty_print_xml(xml_file: Path) -> None:
try:
content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content)
xml_file.write_bytes(dom.toprettyxml(indent=" ", encoding="utf-8"))
except Exception:
pass
def _escape_smart_quotes(xml_file: Path) -> None:
try:
content = xml_file.read_text(encoding="utf-8")
for char, entity in SMART_QUOTE_REPLACEMENTS.items():
content = content.replace(char, entity)
xml_file.write_text(content, encoding="utf-8")
except Exception:
pass
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Unpack an Office file (DOCX, PPTX, XLSX) for editing"
)
parser.add_argument("input_file", help="Office file to unpack")
parser.add_argument("output_directory", help="Output directory")
parser.add_argument(
"--merge-runs",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Merge adjacent runs with identical formatting (DOCX only, default: true)",
)
parser.add_argument(
"--simplify-redlines",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Merge adjacent tracked changes from same author (DOCX only, default: true)",
)
args = parser.parse_args()
_, message = unpack(
args.input_file,
args.output_directory,
merge_runs=args.merge_runs,
simplify_redlines=args.simplify_redlines,
)
print(message)
if "Error" in message:
sys.exit(1)
skills/docx/scripts/office/validate.py
+89 -27
View File
@@ -6,7 +6,7 @@ Usage:
The first argument can be either: The first argument can be either:
- An unpacked directory containing the Office document XML files - An unpacked directory containing the Office document XML files
- A packed Office file (.docx/.pptx/.xlsx) which will be unpacked to a temp directory - A packed Office file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx template) which will be unpacked to a temp directory
Auto-repair fixes: Auto-repair fixes:
- paraId/durableId values that exceed OOXML limits - paraId/durableId values that exceed OOXML limits
@@ -19,20 +19,43 @@ import tempfile
import zipfile import zipfile
from pathlib import Path from pathlib import Path
import defusedxml.ElementTree as ET
from defusedxml.common import DefusedXmlException
from helpers import OOXML_FAMILY, rezip, safe_extract
from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator
WORD_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
def _fail(message: str):
print(f"Error: {message}", file=sys.stderr)
sys.exit(2)
def _has_tracked_changes(unpacked_dir: Path) -> bool:
document = unpacked_dir / "word" / "document.xml"
if not document.is_file():
return False
try:
root = ET.parse(document).getroot()
except (ET.ParseError, DefusedXmlException):
return False
tracked = {f"{{{WORD_NS}}}ins", f"{{{WORD_NS}}}del"}
return any(elem.tag in tracked for elem in root.iter())
def main(): def main():
parser = argparse.ArgumentParser(description="Validate Office document XML files") parser = argparse.ArgumentParser(description="Validate Office document XML files")
parser.add_argument( parser.add_argument(
"path", "path",
help="Path to unpacked directory or packed Office file (.docx/.pptx/.xlsx)", help="Path to unpacked directory or packed Office file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx)",
) )
parser.add_argument( parser.add_argument(
"--original", "--original",
required=False, required=False,
default=None, default=None,
help="Path to original file (.docx/.pptx/.xlsx). If omitted, all XSD errors are reported and redlining validation is skipped.", help="Path to original file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx). If omitted, all XSD errors are reported and redlining validation is skipped.",
) )
parser.add_argument( parser.add_argument(
"-v", "-v",
@@ -43,63 +66,102 @@ def main():
parser.add_argument( parser.add_argument(
"--auto-repair", "--auto-repair",
action="store_true", action="store_true",
help="Automatically repair common issues (hex IDs, whitespace preservation)", help="Automatically repair common issues (hex IDs, whitespace preservation). "
"Modifies the input in place: repairs to a packed file are written back to it.",
) )
parser.add_argument( parser.add_argument(
"--author", "--author",
default="Claude", default=None,
help="Author name for redlining validation (default: Claude)", help="The name you are redlining under. Passing it turns on the "
"tracked-change check: any text differing from --original without a "
"<w:ins>/<w:del> recording it is reported. Untracked edits carry no "
"author, so the check covers them whoever made them — the name marks "
"the run as redlining work and is not used to filter. Requires "
"--original; docx only.",
) )
args = parser.parse_args() args = parser.parse_args()
if args.author is not None and not args.original:
_fail("--author requires --original")
path = Path(args.path) path = Path(args.path)
assert path.exists(), f"Error: {path} does not exist" if not path.exists():
_fail(f"{path} does not exist")
original_file = None original_file = None
if args.original: if args.original:
original_file = Path(args.original) original_file = Path(args.original)
assert original_file.is_file(), f"Error: {original_file} is not a file" if not original_file.is_file():
assert original_file.suffix.lower() in [".docx", ".pptx", ".xlsx"], ( _fail(f"{original_file} is not a file")
f"Error: {original_file} must be a .docx, .pptx, or .xlsx file" if original_file.suffix.lower() not in OOXML_FAMILY:
_fail(f"{original_file} must be one of: {', '.join(sorted(OOXML_FAMILY))}")
family = OOXML_FAMILY.get((original_file or path).suffix.lower())
if family is None:
_fail(
f"Cannot determine file type from {path}. Use --original or provide one of: {', '.join(sorted(OOXML_FAMILY))}."
) )
file_extension = (original_file or path).suffix.lower() if args.author is not None and family != "docx":
assert file_extension in [".docx", ".pptx", ".xlsx"], ( _fail(f"--author only applies to docx files, not {family}")
f"Error: Cannot determine file type from {path}. Use --original or provide a .docx/.pptx/.xlsx file."
)
if path.is_file() and path.suffix.lower() in [".docx", ".pptx", ".xlsx"]: packed_file = None
temp_dir = tempfile.mkdtemp() temp_dir_ctx = None
with zipfile.ZipFile(path, "r") as zf: if path.is_file() and path.suffix.lower() in OOXML_FAMILY:
zf.extractall(temp_dir) packed_file = path
unpacked_dir = Path(temp_dir) temp_dir_ctx = tempfile.TemporaryDirectory()
unpacked_dir = Path(temp_dir_ctx.name)
try:
with zipfile.ZipFile(path, "r") as zf:
safe_extract(zf, unpacked_dir)
except (zipfile.BadZipFile, ValueError, OSError) as e:
_fail(f"cannot unpack {path}: {e}")
else: else:
assert path.is_dir(), f"Error: {path} is not a directory or Office file" if not path.is_dir():
_fail(f"{path} is not a directory or Office file")
unpacked_dir = path unpacked_dir = path
match file_extension: match family:
case ".docx": case "docx":
validators = [ validators = [
DOCXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), DOCXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose),
] ]
if original_file: if args.author is not None:
validators.append( validators.append(
RedliningValidator(unpacked_dir, original_file, verbose=args.verbose, author=args.author) RedliningValidator(unpacked_dir, original_file, verbose=args.verbose)
) )
case ".pptx": elif original_file and _has_tracked_changes(unpacked_dir):
print(
"Note: this document has tracked changes; they were not "
"checked against the original (pass --author to check)."
)
case "pptx":
validators = [ validators = [
PPTXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), PPTXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose),
] ]
case "xlsx":
exts = ", ".join(k for k, v in sorted(OOXML_FAMILY.items()) if v == "xlsx")
print(
f"No XSD schema validation is performed for xlsx-family files ({exts}). "
"For formula-error checking, use scripts/recalc.py instead."
)
sys.exit(0)
case _: case _:
print(f"Error: Validation not supported for file type {file_extension}") print(f"Error: Validation not supported for file type {family}")
sys.exit(1) sys.exit(1)
if args.auto_repair: if args.auto_repair:
total_repairs = sum(v.repair() for v in validators) total_repairs = sum(v.repair() for v in validators)
if total_repairs: if total_repairs:
print(f"Auto-repaired {total_repairs} issue(s)") print(f"Auto-repaired {total_repairs} issue(s)")
if packed_file is not None:
rezip(unpacked_dir, packed_file)
print(f"Wrote repaired file to {packed_file}")
success = all(v.validate() for v in validators) success = all([v.validate() for v in validators])
if temp_dir_ctx is not None:
temp_dir_ctx.cleanup()
if success: if success:
print("All validations PASSED!") print("All validations PASSED!")
skills/docx/scripts/office/validators/base.py
+48 -20
View File
@@ -6,8 +6,20 @@ import re
from pathlib import Path from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
from functools import lru_cache
import lxml.etree import lxml.etree
from helpers import safe_extract
@lru_cache(maxsize=None)
def _load_schema(schema_path: str):
with open(schema_path, "rb") as xsd_file:
xsd_doc = lxml.etree.parse(
xsd_file, parser=lxml.etree.XMLParser(), base_url=schema_path
)
return lxml.etree.XMLSchema(xsd_doc)
class BaseSchemaValidator: class BaseSchemaValidator:
@@ -119,21 +131,28 @@ class BaseSchemaValidator:
try: try:
content = xml_file.read_text(encoding="utf-8") content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content) dom = defusedxml.minidom.parseString(content)
modified = False pending = []
for elem in dom.getElementsByTagName("*"): for elem in dom.getElementsByTagName("*"):
if elem.tagName.endswith(":t") and elem.firstChild: local_name = elem.tagName.rsplit(":", 1)[-1]
text = elem.firstChild.nodeValue if local_name in ("t", "delText", "instrText", "delInstrText"):
if text and (text.startswith((' ', '\t')) or text.endswith((' ', '\t'))): text = "".join(
child.data
for child in elem.childNodes
if child.nodeType in (child.TEXT_NODE, child.CDATA_SECTION_NODE)
)
ws = (" ", "\t", "\n", "\r")
if text and (text.startswith(ws) or text.endswith(ws)):
if elem.getAttribute("xml:space") != "preserve": if elem.getAttribute("xml:space") != "preserve":
elem.setAttribute("xml:space", "preserve") elem.setAttribute("xml:space", "preserve")
text_preview = repr(text[:30]) + "..." if len(text) > 30 else repr(text) text_preview = repr(text[:30]) + "..." if len(text) > 30 else repr(text)
print(f" Repaired: {xml_file.name}: Added xml:space='preserve' to {elem.tagName}: {text_preview}") pending.append(f" Repaired: {xml_file.name}: Added xml:space='preserve' to {elem.tagName}: {text_preview}")
repairs += 1
modified = True
if modified: if pending:
xml_file.write_bytes(dom.toxml(encoding="UTF-8")) xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
for message in pending:
print(message)
repairs += len(pending)
except Exception: except Exception:
pass pass
@@ -212,6 +231,8 @@ class BaseSchemaValidator:
elem.getparent().remove(elem) elem.getparent().remove(elem)
for elem in root.iter(): for elem in root.iter():
if not hasattr(elem, "tag") or callable(elem.tag):
continue
tag = ( tag = (
elem.tag.split("}")[-1].lower() elem.tag.split("}")[-1].lower()
if "}" in elem.tag if "}" in elem.tag
@@ -326,6 +347,8 @@ class BaseSchemaValidator:
namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE}, namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE},
): ):
target = rel.get("Target") target = rel.get("Target")
if rel.get("TargetMode") == "External":
continue
if target and not target.startswith( if target and not target.startswith(
("http", "mailto:") ("http", "mailto:")
): ):
@@ -423,6 +446,8 @@ class BaseSchemaValidator:
r_ns = self.OFFICE_RELATIONSHIPS_NAMESPACE r_ns = self.OFFICE_RELATIONSHIPS_NAMESPACE
rid_attrs_to_check = ["id", "embed", "link"] rid_attrs_to_check = ["id", "embed", "link"]
for elem in xml_root.iter(): for elem in xml_root.iter():
if not hasattr(elem, "tag") or callable(elem.tag):
continue
for attr_name in rid_attrs_to_check: for attr_name in rid_attrs_to_check:
rid_attr = elem.get(f"{{{r_ns}}}{attr_name}") rid_attr = elem.get(f"{{{r_ns}}}{attr_name}")
if not rid_attr: if not rid_attr:
@@ -747,18 +772,16 @@ class BaseSchemaValidator:
return xml_doc return xml_doc
def _validate_single_file_xsd(self, xml_file, base_path): def _preprocess_for_schema(self, xml_doc, relative_path):
schema_path = self._get_schema_path(xml_file) return xml_doc
def _validate_single_file_xsd(self, xml_file, base_path, schema_path=None):
schema_path = schema_path or self._get_schema_path(xml_file)
if not schema_path: if not schema_path:
return None, None return None, None
try: try:
with open(schema_path, "rb") as xsd_file: schema = _load_schema(str(schema_path))
parser = lxml.etree.XMLParser()
xsd_doc = lxml.etree.parse(
xsd_file, parser=parser, base_url=str(schema_path)
)
schema = lxml.etree.XMLSchema(xsd_doc)
with open(xml_file, "r") as f: with open(xml_file, "r") as f:
xml_doc = lxml.etree.parse(f) xml_doc = lxml.etree.parse(f)
@@ -773,6 +796,8 @@ class BaseSchemaValidator:
): ):
xml_doc = self._clean_ignorable_namespaces(xml_doc) xml_doc = self._clean_ignorable_namespaces(xml_doc)
xml_doc = self._preprocess_for_schema(xml_doc, relative_path)
if schema.validate(xml_doc): if schema.validate(xml_doc):
return True, set() return True, set()
else: else:
@@ -784,7 +809,7 @@ class BaseSchemaValidator:
except Exception as e: except Exception as e:
return False, {str(e)} return False, {str(e)}
def _get_original_file_errors(self, xml_file): def _get_original_file_errors(self, xml_file, schema_path=None):
if self.original_file is None: if self.original_file is None:
return set() return set()
@@ -798,8 +823,11 @@ class BaseSchemaValidator:
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir) temp_path = Path(temp_dir)
with zipfile.ZipFile(self.original_file, "r") as zip_ref: try:
zip_ref.extractall(temp_path) with zipfile.ZipFile(self.original_file, "r") as zip_ref:
safe_extract(zip_ref, temp_path)
except (zipfile.BadZipFile, ValueError, OSError):
return set()
original_xml_file = temp_path / relative_path original_xml_file = temp_path / relative_path
@@ -807,7 +835,7 @@ class BaseSchemaValidator:
return set() return set()
is_valid, errors = self._validate_single_file_xsd( is_valid, errors = self._validate_single_file_xsd(
original_xml_file, temp_path original_xml_file, temp_path, schema_path=schema_path
) )
return errors if errors else set() return errors if errors else set()
skills/docx/scripts/office/validators/docx.py
+56 -36
View File
@@ -6,10 +6,13 @@ import random
import re import re
import tempfile import tempfile
import zipfile import zipfile
from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
import lxml.etree import lxml.etree
from helpers import safe_extract
from .base import BaseSchemaValidator from .base import BaseSchemaValidator
@@ -186,7 +189,7 @@ class DOCXSchemaValidator(BaseSchemaValidator):
try: try:
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
with zipfile.ZipFile(original, "r") as zip_ref: with zipfile.ZipFile(original, "r") as zip_ref:
zip_ref.extractall(temp_dir) safe_extract(zip_ref, Path(temp_dir))
doc_xml_path = temp_dir + "/word/document.xml" doc_xml_path = temp_dir + "/word/document.xml"
root = lxml.etree.parse(doc_xml_path).getroot() root = lxml.etree.parse(doc_xml_path).getroot()
@@ -241,9 +244,12 @@ class DOCXSchemaValidator(BaseSchemaValidator):
return True return True
def compare_paragraph_counts(self): def compare_paragraph_counts(self):
original_count = self.count_paragraphs_in_original()
new_count = self.count_paragraphs_in_unpacked() new_count = self.count_paragraphs_in_unpacked()
if self.original_file is None:
print(f"\nParagraphs: {new_count}")
return
original_count = self.count_paragraphs_in_original()
diff = new_count - original_count diff = new_count - original_count
diff_str = f"+{diff}" if diff > 0 else str(diff) diff_str = f"+{diff}" if diff > 0 else str(diff)
print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})") print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})")
@@ -260,9 +266,15 @@ class DOCXSchemaValidator(BaseSchemaValidator):
try: try:
for elem in lxml.etree.parse(str(xml_file)).iter(): for elem in lxml.etree.parse(str(xml_file)).iter():
if val := elem.get(para_id_attr): if val := elem.get(para_id_attr):
if self._parse_id_value(val, base=16) >= 0x80000000: try:
if self._parse_id_value(val, base=16) >= 0x80000000:
errors.append(
f" {xml_file.name}:{elem.sourceline}: paraId={val} >= 0x80000000"
)
except ValueError:
errors.append( errors.append(
f" {xml_file.name}:{elem.sourceline}: paraId={val} >= 0x80000000" f" {xml_file.name}:{elem.sourceline}: "
f"paraId={val} is not valid hex"
) )
if val := elem.get(durable_id_attr): if val := elem.get(durable_id_attr):
@@ -279,13 +291,19 @@ class DOCXSchemaValidator(BaseSchemaValidator):
f"durableId={val} must be decimal in numbering.xml" f"durableId={val} must be decimal in numbering.xml"
) )
else: else:
if self._parse_id_value(val, base=16) >= 0x7FFFFFFF: try:
if self._parse_id_value(val, base=16) >= 0x7FFFFFFF:
errors.append(
f" {xml_file.name}:{elem.sourceline}: "
f"durableId={val} >= 0x7FFFFFFF"
)
except ValueError:
errors.append( errors.append(
f" {xml_file.name}:{elem.sourceline}: " f" {xml_file.name}:{elem.sourceline}: "
f"durableId={val} >= 0x7FFFFFFF" f"durableId={val} is not valid hex"
) )
except Exception: except lxml.etree.XMLSyntaxError:
pass continue
if errors: if errors:
print(f"FAILED - {len(errors)} ID constraint violations:") print(f"FAILED - {len(errors)} ID constraint violations:")
@@ -389,52 +407,54 @@ class DOCXSchemaValidator(BaseSchemaValidator):
return repairs return repairs
def repair_durableId(self) -> int: def repair_durableId(self) -> int:
DURABLE_ID_ATTRS = ("w16cid:durableId", "w16cex:durableId")
repairs = 0 repairs = 0
renames: dict = {}
for xml_file in self.xml_files: for xml_file in self.xml_files:
try: try:
content = xml_file.read_text(encoding="utf-8") content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content) dom = defusedxml.minidom.parseString(content)
is_numbering = xml_file.name == "numbering.xml"
base = 10 if is_numbering else 16
pending = []
seen_in_file = set()
modified = False modified = False
for elem in dom.getElementsByTagName("*"): for elem in dom.getElementsByTagName("*"):
if not elem.hasAttribute("w16cid:durableId"): for attr_name in DURABLE_ID_ATTRS:
continue if not elem.hasAttribute(attr_name):
continue
durable_id = elem.getAttribute("w16cid:durableId") durable_id = elem.getAttribute(attr_name)
needs_repair = False
if xml_file.name == "numbering.xml":
try: try:
needs_repair = ( key = self._parse_id_value(durable_id, base=base)
self._parse_id_value(durable_id, base=10) >= 0x7FFFFFFF needs_repair = key >= 0x7FFFFFFF
)
except ValueError:
needs_repair = True
else:
try:
needs_repair = (
self._parse_id_value(durable_id, base=16) >= 0x7FFFFFFF
)
except ValueError: except ValueError:
key = durable_id
needs_repair = True needs_repair = True
if needs_repair: if needs_repair:
value = random.randint(1, 0x7FFFFFFE) if key in seen_in_file:
if xml_file.name == "numbering.xml": value = random.randint(1, 0x7FFFFFFE)
new_id = str(value) else:
else: seen_in_file.add(key)
new_id = f"{value:08X}" if key not in renames:
renames[key] = random.randint(1, 0x7FFFFFFE)
value = renames[key]
new_id = str(value) if is_numbering else f"{value:08X}"
elem.setAttribute("w16cid:durableId", new_id) elem.setAttribute(attr_name, new_id)
print( pending.append(
f" Repaired: {xml_file.name}: durableId {durable_id} → {new_id}" f" Repaired: {xml_file.name}: durableId {durable_id} → {new_id}"
) )
repairs += 1 modified = True
modified = True
if modified: if modified:
xml_file.write_bytes(dom.toxml(encoding="UTF-8")) xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
for message in pending:
print(message)
repairs += len(pending)
except Exception: except Exception:
pass pass
skills/docx/scripts/office/validators/pptx.py
+173 -7
View File
@@ -3,6 +3,9 @@ Validator for PowerPoint presentation XML files against XSD schemas.
""" """
import re import re
from pathlib import Path
from helpers import opc_target, rels_source_part, safe_extract
from .base import BaseSchemaValidator from .base import BaseSchemaValidator
@@ -57,8 +60,171 @@ class PPTXSchemaValidator(BaseSchemaValidator):
if not self.validate_no_duplicate_slide_layouts(): if not self.validate_no_duplicate_slide_layouts():
all_valid = False all_valid = False
if not self.validate_master_theme_uniqueness():
all_valid = False
if not self.validate_charts():
all_valid = False
if not self.validate_slides():
all_valid = False
return all_valid return all_valid
def _package_map(self) -> dict:
wanted = []
wanted += list(self.unpacked_dir.glob("[[]Content_Types[]].xml"))
wanted += list(self.unpacked_dir.glob("ppt/presentation.xml"))
wanted += list(self.unpacked_dir.glob("ppt/theme/*.xml"))
wanted += list(self.unpacked_dir.glob("ppt/theme/_rels/*.rels"))
wanted += list(self.unpacked_dir.glob("ppt/charts/chart*.xml"))
for group in ("slideMasters", "notesMasters", "handoutMasters"):
wanted += list(self.unpacked_dir.glob(f"ppt/{group}/*.xml"))
wanted += list(self.unpacked_dir.glob(f"ppt/{group}/_rels/*.rels"))
return {
p.relative_to(self.unpacked_dir).as_posix(): p.read_bytes()
for p in wanted
if p.is_file()
}
def validate_master_theme_uniqueness(self):
from helpers.pptx_theme import _NOTES_MASTERS, live_shared_master_themes
shared = live_shared_master_themes(self._package_map())
if shared:
print(f"FAILED - Found {len(shared)} master(s) sharing a theme part:")
for message in shared:
print(f" {message}")
if any(m.startswith(_NOTES_MASTERS) for m in shared):
print(" Fix: in ppt/presentation.xml, move <p:notesMasterIdLst> back to "
"directly after <p:sldIdLst>. PowerPoint reads that happily.")
else:
print(" Fix: give each master its own theme part.")
return False
if self.verbose:
print("PASSED - No master shares a theme part in a way PowerPoint refuses")
return True
def validate_charts(self):
from helpers.pptx_chart import find_chart_problems
problems = find_chart_problems(self._package_map())
if problems:
print(f"FAILED - Found {len(problems)} chart problem(s) PowerPoint rejects:")
for message in problems:
print(f" {message}")
return False
if self.verbose:
print("PASSED - Charts satisfy the constraints PowerPoint enforces")
return True
def _original_slide_defects(self, schema) -> set[str]:
import tempfile
import zipfile
from helpers.pptx_slide import SLIDE_PART_RE, fatal_slide_errors
if self.original_file is None:
return set()
found: set[str] = set()
with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir)
try:
with zipfile.ZipFile(self.original_file, "r") as zf:
safe_extract(zf, temp_path)
except (zipfile.BadZipFile, ValueError, OSError):
return set()
for part in sorted(temp_path.rglob("*.xml")):
relative = part.relative_to(temp_path).as_posix()
if not SLIDE_PART_RE.fullmatch(relative):
continue
ok, errors = self._validate_single_file_xsd(
part.resolve(), temp_path.resolve(), schema_path=schema
)
if ok is None or ok or not errors:
continue
found |= set(fatal_slide_errors(set(errors)))
return found
def validate_slides(self):
from helpers.pptx_slide import (
SLIDE_PART_RE,
fatal_slide_errors,
is_schema_verdict,
)
schema = self.schemas_dir / self.SCHEMA_MAPPINGS["ppt"]
inherited = self._original_slide_defects(schema)
problems: list[str] = []
broken: list[str] = []
for xml_file in self.xml_files:
relative = xml_file.relative_to(self.unpacked_dir).as_posix()
if not SLIDE_PART_RE.fullmatch(relative):
continue
ok, errors = self._validate_single_file_xsd(
xml_file.resolve(), self.unpacked_dir.resolve(), schema_path=schema
)
if ok is None or not errors:
continue
unreadable = [f"{relative}: {e}" for e in errors if not is_schema_verdict(e)]
if unreadable:
broken.extend(unreadable)
continue
if ok:
continue
for message in fatal_slide_errors(set(errors)):
if message in inherited:
continue
problems.append(f"{relative}: {message}")
if broken:
print(f"FAILED - Could not check {len(broken)} slide part(s):")
for message in sorted(broken):
print(f" {message[:240]}")
if problems:
print(f"FAILED - Found {len(problems)} slide problem(s) PowerPoint rejects:")
for message in sorted(problems):
print(f" {message[:240]}")
if broken or problems:
return False
if self.verbose:
print("PASSED - Slide XML has none of the defects PowerPoint refuses")
return True
def _get_schema_path(self, xml_file):
if xml_file.parent.name == "charts" and xml_file.name.startswith("chart"):
return None
return super()._get_schema_path(xml_file)
def _preprocess_for_schema(self, xml_doc, relative_path):
if relative_path.as_posix() != "ppt/presentation.xml":
return xml_doc
root = xml_doc.getroot()
ns = f"{{{self.PRESENTATIONML_NAMESPACE}}}"
notes = root.find(f"{ns}notesMasterIdLst")
slides = root.find(f"{ns}sldIdLst")
if notes is None or slides is None:
return xml_doc
children = list(root)
if children.index(notes) < children.index(slides):
return xml_doc
root.remove(notes)
root.insert(list(root).index(slides), notes)
return xml_doc
def validate_uuid_ids(self): def validate_uuid_ids(self):
import lxml.etree import lxml.etree
@@ -229,17 +395,17 @@ class PPTXSchemaValidator(BaseSchemaValidator):
): ):
rel_type = rel.get("Type", "") rel_type = rel.get("Type", "")
if "notesSlide" in rel_type: if "notesSlide" in rel_type:
target = rel.get("Target", "") part = opc_target(
if target: rel.get("Target", ""),
normalized_target = target.replace("../", "") rels_source_part(rels_file, self.unpacked_dir),
rel.get("TargetMode", ""),
)
if part:
slide_name = rels_file.stem.replace( slide_name = rels_file.stem.replace(
".xml", "" ".xml", ""
) )
if normalized_target not in notes_slide_references: notes_slide_references.setdefault(part, []).append(
notes_slide_references[normalized_target] = []
notes_slide_references[normalized_target].append(
(slide_name, rels_file) (slide_name, rels_file)
) )
skills/docx/scripts/office/validators/redlining.py
+96 -44
View File
@@ -1,5 +1,14 @@
""" """
Validator for tracked changes in Word documents. Validator for tracked changes in Word documents.
Detects untracked edits in word/document.xml: text that differs from the
original without a <w:ins>/<w:del> wrapper recording it. The tracked changes
that are new relative to the original are undone, and the result is compared
against the original; whatever text still differs was edited without being
tracked.
Only the document body is compared. Headers, footers, footnotes and endnotes
are separate parts and are not checked.
""" """
import subprocess import subprocess
@@ -7,14 +16,18 @@ import tempfile
import zipfile import zipfile
from pathlib import Path from pathlib import Path
import defusedxml.ElementTree as ET
from defusedxml.common import DefusedXmlException
from helpers import rendered_text, safe_extract
class RedliningValidator: class RedliningValidator:
def __init__(self, unpacked_dir, original_docx, verbose=False, author="Claude"): def __init__(self, unpacked_dir, original_docx, verbose=False):
self.unpacked_dir = Path(unpacked_dir) self.unpacked_dir = Path(unpacked_dir)
self.original_docx = Path(original_docx) self.original_docx = Path(original_docx)
self.verbose = verbose self.verbose = verbose
self.author = author
self.namespaces = { self.namespaces = {
"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main" "w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
} }
@@ -28,40 +41,12 @@ class RedliningValidator:
print(f"FAILED - Modified document.xml not found at {modified_file}") print(f"FAILED - Modified document.xml not found at {modified_file}")
return False return False
try:
import xml.etree.ElementTree as ET
tree = ET.parse(modified_file)
root = tree.getroot()
del_elements = root.findall(".//w:del", self.namespaces)
ins_elements = root.findall(".//w:ins", self.namespaces)
author_del_elements = [
elem
for elem in del_elements
if elem.get(f"{{{self.namespaces['w']}}}author") == self.author
]
author_ins_elements = [
elem
for elem in ins_elements
if elem.get(f"{{{self.namespaces['w']}}}author") == self.author
]
if not author_del_elements and not author_ins_elements:
if self.verbose:
print(f"PASSED - No tracked changes by {self.author} found.")
return True
except Exception:
pass
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir) temp_path = Path(temp_dir)
try: try:
with zipfile.ZipFile(self.original_docx, "r") as zip_ref: with zipfile.ZipFile(self.original_docx, "r") as zip_ref:
zip_ref.extractall(temp_path) safe_extract(zip_ref, temp_path)
except Exception as e: except Exception as e:
print(f"FAILED - Error unpacking original docx: {e}") print(f"FAILED - Error unpacking original docx: {e}")
return False return False
@@ -74,18 +59,16 @@ class RedliningValidator:
return False return False
try: try:
import xml.etree.ElementTree as ET
modified_tree = ET.parse(modified_file) modified_tree = ET.parse(modified_file)
modified_root = modified_tree.getroot() modified_root = modified_tree.getroot()
original_tree = ET.parse(original_file) original_tree = ET.parse(original_file)
original_root = original_tree.getroot() original_root = original_tree.getroot()
except ET.ParseError as e: except (ET.ParseError, DefusedXmlException) as e:
print(f"FAILED - Error parsing XML files: {e}") print(f"FAILED - Error parsing XML files: {e}")
return False return False
self._remove_author_tracked_changes(original_root) new_changes = self._new_tracked_changes(original_root, modified_root)
self._remove_author_tracked_changes(modified_root) self._remove_tracked_changes(modified_root, new_changes)
modified_text = self._extract_text_content(modified_root) modified_text = self._extract_text_content(modified_root)
original_text = self._extract_text_content(original_root) original_text = self._extract_text_content(original_root)
@@ -98,20 +81,91 @@ class RedliningValidator:
return False return False
if self.verbose: if self.verbose:
print(f"PASSED - All changes by {self.author} are properly tracked") print(
f"PASSED - All {len(new_changes)} change(s) against the original "
"are properly tracked"
)
return True return True
def _tracked_change_elements(self, root):
ins_tag = f"{{{self.namespaces['w']}}}ins"
del_tag = f"{{{self.namespaces['w']}}}del"
return [elem for elem in root.iter() if elem.tag in (ins_tag, del_tag)]
def _rendered_text(self, elem):
preserve = elem.get("{http://www.w3.org/XML/1998/namespace}space") == "preserve"
return rendered_text(elem.text or "", preserve)
def _text_elements(self, elem):
w = self.namespaces["w"]
return [
node
for node in elem.iter()
if node.tag in (f"{{{w}}}t", f"{{{w}}}delText")
]
def _tracked_change_key(self, elem):
w = self.namespaces["w"]
text = "".join(self._rendered_text(node) for node in self._text_elements(elem))
return (elem.tag, elem.get(f"{{{w}}}author"), elem.get(f"{{{w}}}date"), text)
def _new_tracked_changes(self, original_root, modified_root):
original = self._tracked_change_elements(original_root)
modified = self._tracked_change_elements(modified_root)
pool = {}
for elem in original:
pool.setdefault(self._tracked_change_key(elem), []).append(elem)
matched, leftover = set(), []
for elem in modified:
bucket = pool.get(self._tracked_change_key(elem))
if bucket:
matched.add(bucket.pop())
else:
leftover.append(elem)
def group(elem):
return self._tracked_change_key(elem)[:3]
def text_of(elems):
return "".join(self._tracked_change_key(e)[3] for e in elems)
unmatched_original = {}
for elem in original:
if elem not in matched:
unmatched_original.setdefault(group(elem), []).append(elem)
by_group = {}
for elem in leftover:
by_group.setdefault(group(elem), []).append(elem)
new = set()
for key, elems in by_group.items():
rebuilt = text_of(elems)
if rebuilt and rebuilt == text_of(unmatched_original.get(key, [])):
continue
new.update(elems)
return new
def _generate_detailed_diff(self, original_text, modified_text): def _generate_detailed_diff(self, original_text, modified_text):
error_parts = [ error_parts = [
f"FAILED - Document text doesn't match after removing {self.author}'s tracked changes", "FAILED - Document text doesn't match after removing the tracked changes",
"", "",
"Likely causes:", "Likely causes:",
" 1. Modified text inside another author's <w:ins> or <w:del> tags", " 1. Modified text inside another author's <w:ins> or <w:del> tags",
" 2. Made edits without proper tracked changes", " 2. Made edits without proper tracked changes",
" 3. Didn't nest <w:del> inside <w:ins> when deleting another's insertion", " 3. Didn't nest <w:del> inside <w:ins> when deleting another's insertion",
" 4. Rewrote another author's <w:ins>/<w:del> and changed its text on",
" the way. A tracked change from the original is recognised by its",
" author, date and text; anything that doesn't reproduce one exactly",
" reads as new, and the text it carried is reported missing.",
"", "",
"For pre-redlined documents, use correct patterns:", "For pre-redlined documents, use correct patterns:",
" - To reject another's INSERTION: Nest <w:del> inside their <w:ins>", " - To reject another's INSERTION: Nest <w:del> inside their <w:ins>",
" - To reject PART of one: nest <w:del> around only the runs you reject.",
" Their <w:ins> may be split around it, so long as the pieces keep",
" their author and date and still spell out the same text.",
" - To restore another's DELETION: Add new <w:ins> AFTER their <w:del>", " - To restore another's DELETION: Add new <w:ins> AFTER their <w:del>",
"", "",
] ]
@@ -195,15 +249,14 @@ class RedliningValidator:
return None return None
def _remove_author_tracked_changes(self, root): def _remove_tracked_changes(self, root, targets):
ins_tag = f"{{{self.namespaces['w']}}}ins" ins_tag = f"{{{self.namespaces['w']}}}ins"
del_tag = f"{{{self.namespaces['w']}}}del" del_tag = f"{{{self.namespaces['w']}}}del"
author_attr = f"{{{self.namespaces['w']}}}author"
for parent in root.iter(): for parent in root.iter():
to_remove = [] to_remove = []
for child in parent: for child in parent:
if child.tag == ins_tag and child.get(author_attr) == self.author: if child.tag == ins_tag and child in targets:
to_remove.append(child) to_remove.append(child)
for elem in to_remove: for elem in to_remove:
parent.remove(elem) parent.remove(elem)
@@ -214,7 +267,7 @@ class RedliningValidator:
for parent in root.iter(): for parent in root.iter():
to_process = [] to_process = []
for child in parent: for child in parent:
if child.tag == del_tag and child.get(author_attr) == self.author: if child.tag == del_tag and child in targets:
to_process.append((child, list(parent).index(child))) to_process.append((child, list(parent).index(child)))
for del_elem, del_index in reversed(to_process): for del_elem, del_index in reversed(to_process):
@@ -234,8 +287,7 @@ class RedliningValidator:
for p_elem in root.findall(f".//{p_tag}"): for p_elem in root.findall(f".//{p_tag}"):
text_parts = [] text_parts = []
for t_elem in p_elem.findall(f".//{t_tag}"): for t_elem in p_elem.findall(f".//{t_tag}"):
if t_elem.text: text_parts.append(self._rendered_text(t_elem))
text_parts.append(t_elem.text)
paragraph_text = "".join(text_parts) paragraph_text = "".join(text_parts)
if paragraph_text: if paragraph_text:
paragraphs.append(paragraph_text) paragraphs.append(paragraph_text)
skills/pptx/SKILL.md
+102 -96
View File
@@ -1,52 +1,81 @@
--- ---
name: pptx name: pptx
description: "Use this skill any time a .pptx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates, layouts, speaker notes, or comments. Trigger whenever the user mentions \"deck,\" \"slides,\" \"presentation,\" or references a .pptx filename, regardless of what they plan to do with the content afterward. If a .pptx file needs to be opened, created, or touched, use this skill." description: "Use this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user mentions \"deck,\" \"slides,\" \"presentation,\" or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. If a .pptx or .potx file needs to be opened, created, or touched, use this skill."
license: Proprietary. LICENSE.txt has complete terms license: Proprietary. LICENSE.txt has complete terms
--- ---
# PPTX Skill # PPTX creation, editing, and analysis
## Quick Reference A `.pptx` is a ZIP archive of XML files. Choose your approach by task:
| Task | Guide | | Task | Approach |
|------|-------| |---|---|
| Read/analyze content | `python -m markitdown presentation.pptx` | | **Create** a new deck | Write a `pptxgenjs` script — see gotchas below |
| Edit or create from template | Read [editing.md](editing.md) | | **Edit** an existing deck, or build from a template | unzip → edit `ppt/slides/slideN.xml` → zip |
| Create from scratch | Read [pptxgenjs.md](pptxgenjs.md) | | **Read** content | `markitdown deck.pptx` (one block per slide under `<!-- Slide number: N -->` markers); visual grid: `python scripts/thumbnail.py deck.pptx` |
--- ## Scripts
## Reading Content Paths are relative to this skill's directory. Everything else is plain Python, `node`, or shell.
| Script | What it does |
|---|---|
| `scripts/thumbnail.py deck.pptx [prefix]` | Labeled grid of every slide, for picking template layouts. `.pptx` only. Pass `prefix` — it defaults to `thumbnails`, which overwrites the grids of any other deck done in the same directory |
| `scripts/add_slide.py unpacked/ slide2.xml [--after slideN.xml]` | Duplicate a slide (or a `slideLayoutN.xml`) with all the package bookkeeping. Also takes a `.pptx` directly with `-o out.pptx` |
| `scripts/clean.py unpacked/` | Delete slides, media, and rels no longer referenced. Run **after** `<p:sldIdLst>` is final |
| `scripts/office/validate.py deck.pptx [--original src.pptx]` | Schema, relationship, content-type, chart and slide checks; each failure names its fix. Pass `--original` for any template-derived deck — it baselines the schema checks against the template, so the template's own XSD errors don't read as yours |
| `scripts/office/soffice.py --headless --convert-to pdf deck.pptx` | LibreOffice wrapper — bare `soffice` hangs in this sandbox |
## Creating with pptxgenjs — gotchas
`pptxgenjs` is preinstalled — do not run `npm install` first; write the script and `require('pptxgenjs')` directly. Only if that require fails: `npm install pptxgenjs`. The model knows the API; these are the footguns:
- **Set `pres.layout` before adding slides.** The default canvas is `LAYOUT_16x9` = **10" × 5.625"**, not 13.3" wide. Coordinates past the edge are written, not clamped — the shape just isn't on the slide. (`LAYOUT_WIDE` is 13.3" × 7.5".)
- **Hex colors: never `#`, never 8 digits.** `color: "FF0000"`. Both `"#FF0000"` and alpha baked into the hex (`"00000020"`) **corrupt the file**. For translucency: `transparency: 0-100` on fills and images, `opacity: 0.0-1.0` on shadows — each is silently ignored on the other.
- **pptxgenjs mutates option objects in place** (converts values to EMU on first use). Never share one `shadow`/options object across two `add*` calls — build a fresh object each time.
- **Shadow `offset` must be ≥ 0** — a negative offset corrupts the file. To cast a shadow upward, use `angle: 270` with a positive offset.
- **`letterSpacing` is silently ignored** — the real option is `charSpacing`.
- **Lists:** `bullet: true` on each item, never a literal `•` (renders double bullets). Set `breakLine: true` on every array item except the last. Space bulleted paragraphs with `paraSpaceAfter`, not `lineSpacing` (huge gaps).
- **One `new pptxgen()` per output file** — never reuse an instance.
- **`rectRadius` only works on `ROUNDED_RECTANGLE`**, not `RECTANGLE`.
- **Gradient fills aren't supported** — use a gradient image as the background instead.
- **Text boxes have built-in internal padding** — set `margin: 0` whenever text must align with a shape, line, or icon at the same x.
- **Speaker notes go in `slide.addNotes("...")`** (plain text, once per slide), never in a text box on the slide.
- **Keep charts native.** Use `addChart()` for everything PowerPoint can chart (pass an array of `{type, data, options}` for combos). For PowerPoint-native features the library doesn't expose (trendlines, error bars), compute the extra series yourself or post-process the generated OOXML — do not fall back to a rendered image. Only chart types PowerPoint has no native form for (Sankey, network, chord) go in as images.
- **Default charts render bare** — no title, no data labels, dated palette. Set `showTitle` + `title`, `showValue: true` + `dataLabelPosition`, `chartColors: [...]` from your palette, and quiet the frame (`catAxisLabelColor`/`valAxisLabelColor`, `valGridLine: { color, size }`, `catGridLine: { style: "none" }`, `showLegend: false` for a single series).
- **On a stacked bar or column chart, `dataLabelPosition` must be `ctr`, `inEnd`, or `inBase`.** `outEnd` **corrupts the file**.
- **A combo series using `secondaryValAxis`/`secondaryCatAxis` needs both `valAxes` and `catAxes` on the chart options, two entries each.** Without them pptxgenjs writes axis *ids* it never declares, and PowerPoint **discards that chart** and reports the file as corrupt. Supplying only `valAxes` is not enough.
- **After `writeFile()`, run `python scripts/office/validate.py deck.pptx`.** It reports the two chart faults above and the slide-XML defects PowerPoint refuses, and names the fix for each. Fix them in your generator, not by hand-editing the packed XML.
- **Never reorder the children of `<p:presentation>`.** pptxgenjs writes `<p:notesMasterIdLst>` right after `<p:sldIdLst>` and points both masters at one theme part. PowerPoint reads that happily — move the element and the same deck becomes unopenable.
- **Icons:** render `react-icons` to SVG (`ReactDOMServer.renderToStaticMarkup`), rasterize with `sharp` at ≥256px, and insert via `addImage({ data: "image/png;base64," + buf.toString("base64") })` — the `image/png;base64,` prefix is required (`react-icons`, `react`, `react-dom`, and `sharp` are preinstalled — `npm install react-icons react react-dom sharp` only if a require fails).
## Editing existing decks and templates
Pick layouts first: `python scripts/thumbnail.py template.pptx template-thumbs` writes a labeled grid of every slide and prints the file(s) it created — `template-thumbs.jpg`, split into `template-thumbs-N.jpg` past 12 slides. **Always pass that second argument, named after the deck.** It defaults to `thumbnails`, so two decks thumbnailed in one directory silently overwrite each other's grids — the first deck's are simply gone (template analysis only — visual QA needs the full-resolution renders from [Converting to Images](#converting-to-images); it only accepts `.pptx`, so copy a `.potx` to a `.pptx` name first). Use it with `markitdown` to map each content section onto a template slide, and vary the layouts — don't put every section on the same title-and-bullets slide.
```bash ```bash
# Text extraction python3 -c "import sys,zipfile; zipfile.ZipFile(sys.argv[1]).extractall('unpacked')" deck.pptx
python -m markitdown presentation.pptx python scripts/add_slide.py unpacked/ slide2.xml --after slide2.xml # duplicate a slide (or slideLayoutN.xml); prints the new slide's path
# reorder / delete slides = edit <p:sldIdLst> in ppt/presentation.xml
# Visual overview python scripts/clean.py unpacked/ # after deletions: removes orphaned slides, media, rels
python scripts/thumbnail.py presentation.pptx # edit slide content in ppt/slides/slideN.xml
(cd unpacked && rm -f ../out.pptx && zip -Xr ../out.pptx .) # zip from INSIDE the dir; rm first or deleted parts survive
# Raw XML python scripts/office/validate.py out.pptx --original deck.pptx
python scripts/office/unpack.py presentation.pptx unpacked/
``` ```
--- - **Do all structural work — add, delete, reorder — before editing any slide's content.** `add_slide.py` copies a slide file verbatim, so duplicating after you edit clones the edited content; and `clean.py` deletes any slide missing from `<p:sldIdLst>`, including one you just wrote.
- **Never copy a slide file by hand** — `add_slide.py` does every registration a new slide needs and reports what it made (`Created ppt/slides/slide17.xml from slide2.xml`). It also works directly on a file: `add_slide.py deck.pptx slide2.xml -o out.pptx` — **pass `-o`, or it rewrites the input deck in place.** A duplicated slide still *references* its source's chart/SmartArt/embedded-object parts rather than cloning them, so editing one slide's chart changes the other's.
- **If you use `python-pptx`**, three things it won't do: duplicate a slide (its only entry point is `add_slide(layout)`), preserve formatting through `text_frame.text = "..."` (that collapses the paragraph to a single unstyled run — assign `run.text` instead), or read the SVG/EMF most template art uses (`add_picture` raises `UnidentifiedImageError`).
- Legacy `.ppt` must be converted first: `python scripts/office/soffice.py --headless --convert-to pptx file.ppt`. `.potx` templates unpack and pack identically — keep the `.potx` extension on the output.
- To reuse a template icon or image, duplicate a slide or layout that already contains it.
## Editing Workflow When filling in a template:
**Read [editing.md](editing.md) for full details.** - If you script an XML transform, parse with `defusedxml.minidom` — round-tripping OOXML through `xml.etree.ElementTree` rewrites namespace prefixes and corrupts the deck.
- **Template slots ≠ source items.** If the template shows 4 team members and you have 3, delete the 4th member's entire group (image + text boxes), not just its text — then check for orphaned visuals in QA.
1. Analyze template with `thumbnail.py` - One `<a:p>` per list item — never concatenate items into a single paragraph. Copy the sibling `<a:pPr>` to preserve spacing, and put `b="1"` on the `<a:rPr>` of titles, section headers, and inline labels (`Status:`, `Owner:`).
2. Unpack → manipulate slides → edit content → clean → pack - Let bullets inherit from the layout; only add `<a:buChar>`, `<a:buAutoNum>` (numbered), or `<a:buNone>` to override — never a literal `•` in the text.
- Text with leading or trailing spaces needs `xml:space="preserve"` on its `<a:t>`.
---
## Creating from Scratch
**Read [pptxgenjs.md](pptxgenjs.md) for full details.**
Use when no template or reference presentation is available.
---
## Design Ideas ## Design Ideas
@@ -57,7 +86,7 @@ Use when no template or reference presentation is available.
- **Pick a bold, content-informed color palette**: The palette should feel designed for THIS topic. If swapping your colors into a completely different presentation would still "work," you haven't made specific enough choices. - **Pick a bold, content-informed color palette**: The palette should feel designed for THIS topic. If swapping your colors into a completely different presentation would still "work," you haven't made specific enough choices.
- **Dominance over equality**: One color should dominate (60-70% visual weight), with 1-2 supporting tones and one sharp accent. Never give all colors equal weight. - **Dominance over equality**: One color should dominate (60-70% visual weight), with 1-2 supporting tones and one sharp accent. Never give all colors equal weight.
- **Dark/light contrast**: Dark backgrounds for title + conclusion slides, light for content ("sandwich" structure). Or commit to dark throughout for a premium feel. - **Dark/light contrast**: Dark backgrounds for title + conclusion slides, light for content ("sandwich" structure). Or commit to dark throughout for a premium feel.
- **Commit to a visual motif**: Pick ONE distinctive element and repeat it — rounded image frames, icons in colored circles, thick single-side borders. Carry it across every slide. - **Commit to a visual motif**: Pick ONE distinctive element and repeat it — rounded image frames, icons in colored circles. Carry it across every slide. **Do not use a color bar or accent stripe as your motif** (see Avoid list).
### Color Palettes ### Color Palettes
@@ -97,18 +126,13 @@ Choose colors that match your topic — don't default to generic blue. Use these
### Typography ### Typography
**Choose an interesting font pairing** — don't default to Arial. Pick a header font with personality and pair it with a clean body font. **Font names you write into the .pptx are rendered by the user's PowerPoint, not by this environment.** Your visual QA renders via LibreOffice, which substitutes fonts it doesn't have — and for some fonts the substitute has different widths, so your QA preview can show text overflow (or fit) that the real deck won't have. To keep your QA trustworthy:
| Header Font | Body Font | - **Safe fonts** (render true-to-width in QA *and* ship with Office): **Arial, Calibri, Cambria, Times New Roman, Courier New, Bookman Old Style, Century Schoolbook**. Use these for body text and anything where fit matters.
|-------------|-----------| - **Headers with personality at zero QA risk**: pair a safe-list serif header (Cambria, Bookman Old Style, Century Schoolbook) with a safe-list sans body (Calibri or Arial). You get visual contrast without giving up reliable overflow checks.
| Georgia | Calibri | - **If the user asks for a font outside the safe list** (e.g. Georgia or Trebuchet MS): use it where the user asked, but size those containers with extra slack (~10%) and don't trust QA text-fit on those elements — the preview of that font is approximate. If the user hasn't specified, prefer safe-list fonts for body text.
| Arial Black | Arial | - **QA-unreliable fonts** (substitute has different widths — overflow checks can be wrong): Georgia, Trebuchet MS, Impact, Arial Black, Garamond, Consolas, Palatino Linotype. Calibri Light substitution varies by environment; treat as QA-unreliable. Fine for titles/accents with slack; don't trust QA text-fit on these.
| Calibri | Calibri Light | - **Never default to Aptos** — Office's post-2023 default has no metric-compatible substitute here *and* is missing from older Office installs, so it's unreliable on both ends.
| Cambria | Calibri |
| Trebuchet MS | Calibri |
| Impact | Arial |
| Palatino | Garamond |
| Consolas | Calibri |
| Element | Size | | Element | Size |
|---------|------| |---------|------|
@@ -135,19 +159,18 @@ Choose colors that match your topic — don't default to generic blue. Use these
- **Don't forget text box padding** — when aligning lines or shapes with text edges, set `margin: 0` on the text box or offset the shape to account for padding - **Don't forget text box padding** — when aligning lines or shapes with text edges, set `margin: 0` on the text box or offset the shape to account for padding
- **Don't use low-contrast elements** — icons AND text need strong contrast against the background; avoid light text on light backgrounds or dark text on dark backgrounds - **Don't use low-contrast elements** — icons AND text need strong contrast against the background; avoid light text on light backgrounds or dark text on dark backgrounds
- **NEVER use accent lines under titles** — these are a hallmark of AI-generated slides; use whitespace or background color instead - **NEVER use accent lines under titles** — these are a hallmark of AI-generated slides; use whitespace or background color instead
- **NEVER add decorative color bars or accent stripes** — this includes: header/footer bars spanning the slide width, vertical sidebar stripes down one edge of the slide, thin accent stripes along one edge of a card or content block, and "single-side borders" on rectangles. These read as AI-generated filler. If you want to set a card apart, use a subtle background tint, a drop shadow, or an icon — not an edge stripe.
--- - **Don't default to cream/beige backgrounds** — when no background is specified, use white (`FFFFFF`) or the user's brand palette; avoid warm-neutral defaults like `F5F5DC`, `FAF0E6`, `FAEBD7`, `FFF8E1`
- **Don't ship text that overflows its shape** — if text doesn't fit, reduce font size, split across slides, or enlarge the container; never leave content cut off or spilling past bounds
## QA (Required) ## QA (Required)
**Assume there are problems. Your job is to find them.** Your first render usually has a few real issues — overlaps, overflow, misalignment. Find and fix those, re-render only the slides you changed, and stop.
Your first render is almost never correct. Approach QA as a bug hunt, not a confirmation step. If you found zero issues on first inspection, you weren't looking hard enough.
### Content QA ### Content QA
```bash ```bash
python -m markitdown output.pptx markitdown output.pptx
``` ```
Check for missing content, typos, wrong order. Check for missing content, typos, wrong order.
@@ -155,78 +178,61 @@ Check for missing content, typos, wrong order.
**When using templates, check for leftover placeholder text:** **When using templates, check for leftover placeholder text:**
```bash ```bash
python -m markitdown output.pptx | grep -iE "xxxx|lorem|ipsum|this.*(page|slide).*layout" markitdown output.pptx | grep -iE "\bx{3,}\b|lorem|ipsum|\bTODO|\[insert|this.*(page|slide).*layout"
``` ```
If grep returns results, fix them before declaring success. If grep returns results, fix them before declaring success.
### File QA (required)
```bash
python scripts/office/validate.py output.pptx # built from scratch
python scripts/office/validate.py output.pptx --original src.pptx # built from a template
```
**If the deck came from a template, always pass `--original`.** A template may itself
contain parts the XSD rejects, so a bare run can report failures you never caused — and
a genuine regression can hide among them. `--original` baselines
the schema and slide checks against the template, suppressing errors it already had.
The structural checks — relationships, content types, charts — ignore `--original` and
report template-inherited problems either way, so read those on their own merits.
pptxgenjs emits chart XML PowerPoint refuses to open, and every other tool
accepts: python-pptx opens those decks, LibreOffice renders them, the XSD
passes them. Every failure names its fix. Fix it in the generator and rebuild.
### Visual QA ### Visual QA
**⚠️ USE SUBAGENTS** — even for 2-3 slides. You've been staring at the code and will see what you expect, not what's there. Subagents have fresh eyes. Convert the slides to images (see [Converting to Images](#converting-to-images)) and inspect every one. After staring at the generating code you tend to see what you expect rather than what rendered, so look at the images fresh (a subagent works well for this if you have one). User-visible defects to look for:
Convert slides to images (see [Converting to Images](#converting-to-images)), then use this prompt: - **Text overflow or text cut off at a box or slide boundary — check this first.** It is the most common defect and always user-visible. (For a font the previewer renders unreliably per Typography, the preview is approximate: trust the ~10% slack you left, not its apparent fit.)
```
Visually inspect these slides. Assume there are issues — find them.
Look for:
- Overlapping elements (text through shapes, lines through words, stacked elements) - Overlapping elements (text through shapes, lines through words, stacked elements)
- Text overflow or cut off at edges/box boundaries
- Decorative lines positioned for single-line text but title wrapped to two lines
- Source citations or footers colliding with content above - Source citations or footers colliding with content above
- Elements too close (< 0.3" gaps) or cards/sections nearly touching - Elements too close (< 0.3" gaps) or cards/sections nearly touching
- Uneven gaps (large empty area in one place, cramped in another) - Uneven gaps (large empty area in one place, cramped in another)
- Insufficient margin from slide edges (< 0.5") - Insufficient margin from slide edges (< 0.5")
- Columns or similar elements not aligned consistently - Columns or similar elements not aligned consistently
- Low-contrast text (e.g., light gray text on cream-colored background) - Low-contrast text (e.g., light gray text on cream-colored background)
- Template decoration mispositioned after text replacement — e.g., a title underline positioned for one line, but the replaced title wrapped to two
- Low-contrast icons (e.g., dark icons on dark backgrounds without a contrasting circle) - Low-contrast icons (e.g., dark icons on dark backgrounds without a contrasting circle)
- Text boxes too narrow causing excessive wrapping - Text boxes too narrow causing excessive wrapping
- Leftover placeholder content - Leftover placeholder content
For each slide, list issues or areas of concern, even if minor.
Read and analyze these images:
1. /path/to/slide-01.jpg (Expected: [brief description])
2. /path/to/slide-02.jpg (Expected: [brief description])
Report ALL issues found, including minor ones.
```
### Verification Loop
1. Generate slides → Convert to images → Inspect
2. **List issues found** (if none found, look again more critically)
3. Fix issues
4. **Re-verify affected slides** — one fix often creates another problem
5. Repeat until a full pass reveals no new issues
**Do not declare success until you've completed at least one fix-and-verify cycle.**
---
## Converting to Images ## Converting to Images
Convert presentations to individual slide images for visual inspection: Convert presentations to individual slide images for visual inspection:
```bash ```bash
python scripts/office/soffice.py --headless --convert-to pdf output.pptx python scripts/office/soffice.py --headless --convert-to pdf output.pptx
rm -f slide-*.jpg
pdftoppm -jpeg -r 150 output.pdf slide pdftoppm -jpeg -r 150 output.pdf slide
ls -1 "$PWD"/slide-*.jpg
``` ```
This creates `slide-01.jpg`, `slide-02.jpg`, etc. **Pass the absolute paths printed above directly to the view tool.** The `rm` clears stale images from prior runs. `pdftoppm` zero-pads based on page count: `slide-1.jpg` for decks under 10 pages, `slide-01.jpg` for 10-99, `slide-001.jpg` for 100+.
To re-render specific slides after fixes: **After fixes, rerun all four commands above** — the PDF must be regenerated from the edited `.pptx` before `pdftoppm` can reflect your changes.
```bash
pdftoppm -jpeg -r 150 -f N -l N output.pdf slide-fixed
```
---
## Dependencies ## Dependencies
- `pip install "markitdown[pptx]"` - text extraction `pptxgenjs` (npm, preinstalled — install only if `require('pptxgenjs')` fails) · `markitdown[pptx]`, `Pillow`, `defusedxml`, `lxml` (pip — text dump, thumbnail, clean, validate) · LibreOffice (`soffice`, auto-configured for sandboxed environments via `scripts/office/soffice.py`) · `pdftoppm` (Poppler)
- `pip install Pillow` - thumbnail grids
- `npm install -g pptxgenjs` - creating from scratch
- LibreOffice (`soffice`) - PDF conversion (auto-configured for sandboxed environments via `scripts/office/soffice.py`)
- Poppler (`pdftoppm`) - PDF to images
skills/pptx/editing.md
-205
View File
@@ -1,205 +0,0 @@
# Editing Presentations
## Template-Based Workflow
When using an existing presentation as a template:
1. **Analyze existing slides**:
```bash
python scripts/thumbnail.py template.pptx
python -m markitdown template.pptx
```
Review `thumbnails.jpg` to see layouts, and markitdown output to see placeholder text.
2. **Plan slide mapping**: For each content section, choose a template slide.
⚠️ **USE VARIED LAYOUTS** — monotonous presentations are a common failure mode. Don't default to basic title + bullet slides. Actively seek out:
- Multi-column layouts (2-column, 3-column)
- Image + text combinations
- Full-bleed images with text overlay
- Quote or callout slides
- Section dividers
- Stat/number callouts
- Icon grids or icon + text rows
**Avoid:** Repeating the same text-heavy layout for every slide.
Match content type to layout style (e.g., key points → bullet slide, team info → multi-column, testimonials → quote slide).
3. **Unpack**: `python scripts/office/unpack.py template.pptx unpacked/`
4. **Build presentation** (do this yourself, not with subagents):
- Delete unwanted slides (remove from `<p:sldIdLst>`)
- Duplicate slides you want to reuse (`add_slide.py`)
- Reorder slides in `<p:sldIdLst>`
- **Complete all structural changes before step 5**
5. **Edit content**: Update text in each `slide{N}.xml`.
**Use subagents here if available** — slides are separate XML files, so subagents can edit in parallel.
6. **Clean**: `python scripts/clean.py unpacked/`
7. **Pack**: `python scripts/office/pack.py unpacked/ output.pptx --original template.pptx`
---
## Scripts
| Script | Purpose |
|--------|---------|
| `unpack.py` | Extract and pretty-print PPTX |
| `add_slide.py` | Duplicate slide or create from layout |
| `clean.py` | Remove orphaned files |
| `pack.py` | Repack with validation |
| `thumbnail.py` | Create visual grid of slides |
### unpack.py
```bash
python scripts/office/unpack.py input.pptx unpacked/
```
Extracts PPTX, pretty-prints XML, escapes smart quotes.
### add_slide.py
```bash
python scripts/add_slide.py unpacked/ slide2.xml # Duplicate slide
python scripts/add_slide.py unpacked/ slideLayout2.xml # From layout
```
Prints `<p:sldId>` to add to `<p:sldIdLst>` at desired position.
### clean.py
```bash
python scripts/clean.py unpacked/
```
Removes slides not in `<p:sldIdLst>`, unreferenced media, orphaned rels.
### pack.py
```bash
python scripts/office/pack.py unpacked/ output.pptx --original input.pptx
```
Validates, repairs, condenses XML, re-encodes smart quotes.
### thumbnail.py
```bash
python scripts/thumbnail.py input.pptx [output_prefix] [--cols N]
```
Creates `thumbnails.jpg` with slide filenames as labels. Default 3 columns, max 12 per grid.
**Use for template analysis only** (choosing layouts). For visual QA, use `soffice` + `pdftoppm` to create full-resolution individual slide images—see SKILL.md.
---
## Slide Operations
Slide order is in `ppt/presentation.xml` → `<p:sldIdLst>`.
**Reorder**: Rearrange `<p:sldId>` elements.
**Delete**: Remove `<p:sldId>`, then run `clean.py`.
**Add**: Use `add_slide.py`. Never manually copy slide files—the script handles notes references, Content_Types.xml, and relationship IDs that manual copying misses.
---
## Editing Content
**Subagents:** If available, use them here (after completing step 4). Each slide is a separate XML file, so subagents can edit in parallel. In your prompt to subagents, include:
- The slide file path(s) to edit
- **"Use the Edit tool for all changes"**
- The formatting rules and common pitfalls below
For each slide:
1. Read the slide's XML
2. Identify ALL placeholder content—text, images, charts, icons, captions
3. Replace each placeholder with final content
**Use the Edit tool, not sed or Python scripts.** The Edit tool forces specificity about what to replace and where, yielding better reliability.
### Formatting Rules
- **Bold all headers, subheadings, and inline labels**: Use `b="1"` on `<a:rPr>`. This includes:
- Slide titles
- Section headers within a slide
- Inline labels like (e.g.: "Status:", "Description:") at the start of a line
- **Never use unicode bullets (•)**: Use proper list formatting with `<a:buChar>` or `<a:buAutoNum>`
- **Bullet consistency**: Let bullets inherit from the layout. Only specify `<a:buChar>` or `<a:buNone>`.
---
## Common Pitfalls
### Template Adaptation
When source content has fewer items than the template:
- **Remove excess elements entirely** (images, shapes, text boxes), don't just clear text
- Check for orphaned visuals after clearing text content
- Run visual QA to catch mismatched counts
When replacing text with different length content:
- **Shorter replacements**: Usually safe
- **Longer replacements**: May overflow or wrap unexpectedly
- Test with visual QA after text changes
- Consider truncating or splitting content to fit the template's design constraints
**Template slots ≠ Source items**: If template has 4 team members but source has 3 users, delete the 4th member's entire group (image + text boxes), not just the text.
### Multi-Item Content
If source has multiple items (numbered lists, multiple sections), create separate `<a:p>` elements for each — **never concatenate into one string**.
**❌ WRONG** — all items in one paragraph:
```xml
<a:p>
<a:r><a:rPr .../><a:t>Step 1: Do the first thing. Step 2: Do the second thing.</a:t></a:r>
</a:p>
```
**✅ CORRECT** — separate paragraphs with bold headers:
```xml
<a:p>
<a:pPr algn="l"><a:lnSpc><a:spcPts val="3919"/></a:lnSpc></a:pPr>
<a:r><a:rPr lang="en-US" sz="2799" b="1" .../><a:t>Step 1</a:t></a:r>
</a:p>
<a:p>
<a:pPr algn="l"><a:lnSpc><a:spcPts val="3919"/></a:lnSpc></a:pPr>
<a:r><a:rPr lang="en-US" sz="2799" .../><a:t>Do the first thing.</a:t></a:r>
</a:p>
<a:p>
<a:pPr algn="l"><a:lnSpc><a:spcPts val="3919"/></a:lnSpc></a:pPr>
<a:r><a:rPr lang="en-US" sz="2799" b="1" .../><a:t>Step 2</a:t></a:r>
</a:p>
<!-- continue pattern -->
```
Copy `<a:pPr>` from the original paragraph to preserve line spacing. Use `b="1"` on headers.
### Smart Quotes
Handled automatically by unpack/pack. But the Edit tool converts smart quotes to ASCII.
**When adding new text with quotes, use XML entities:**
```xml
<a:t>the &#x201C;Agreement&#x201D;</a:t>
```
| Character | Name | Unicode | XML Entity |
|-----------|------|---------|------------|
| `“` | Left double quote | U+201C | `&#x201C;` |
| `”` | Right double quote | U+201D | `&#x201D;` |
| `‘` | Left single quote | U+2018 | `&#x2018;` |
| `’` | Right single quote | U+2019 | `&#x2019;` |
### Other
- **Whitespace**: Use `xml:space="preserve"` on `<a:t>` with leading/trailing spaces
- **XML parsing**: Use `defusedxml.minidom`, not `xml.etree.ElementTree` (corrupts namespaces)
skills/pptx/pptxgenjs.md
-420
View File
@@ -1,420 +0,0 @@
# PptxGenJS Tutorial
## Setup & Basic Structure
```javascript
const pptxgen = require("pptxgenjs");
let pres = new pptxgen();
pres.layout = 'LAYOUT_16x9'; // or 'LAYOUT_16x10', 'LAYOUT_4x3', 'LAYOUT_WIDE'
pres.author = 'Your Name';
pres.title = 'Presentation Title';
let slide = pres.addSlide();
slide.addText("Hello World!", { x: 0.5, y: 0.5, fontSize: 36, color: "363636" });
pres.writeFile({ fileName: "Presentation.pptx" });
```
## Layout Dimensions
Slide dimensions (coordinates in inches):
- `LAYOUT_16x9`: 10" × 5.625" (default)
- `LAYOUT_16x10`: 10" × 6.25"
- `LAYOUT_4x3`: 10" × 7.5"
- `LAYOUT_WIDE`: 13.3" × 7.5"
---
## Text & Formatting
```javascript
// Basic text
slide.addText("Simple Text", {
x: 1, y: 1, w: 8, h: 2, fontSize: 24, fontFace: "Arial",
color: "363636", bold: true, align: "center", valign: "middle"
});
// Character spacing (use charSpacing, not letterSpacing which is silently ignored)
slide.addText("SPACED TEXT", { x: 1, y: 1, w: 8, h: 1, charSpacing: 6 });
// Rich text arrays
slide.addText([
{ text: "Bold ", options: { bold: true } },
{ text: "Italic ", options: { italic: true } }
], { x: 1, y: 3, w: 8, h: 1 });
// Multi-line text (requires breakLine: true)
slide.addText([
{ text: "Line 1", options: { breakLine: true } },
{ text: "Line 2", options: { breakLine: true } },
{ text: "Line 3" } // Last item doesn't need breakLine
], { x: 0.5, y: 0.5, w: 8, h: 2 });
// Text box margin (internal padding)
slide.addText("Title", {
x: 0.5, y: 0.3, w: 9, h: 0.6,
margin: 0 // Use 0 when aligning text with other elements like shapes or icons
});
```
**Tip:** Text boxes have internal margin by default. Set `margin: 0` when you need text to align precisely with shapes, lines, or icons at the same x-position.
---
## Lists & Bullets
```javascript
// ✅ CORRECT: Multiple bullets
slide.addText([
{ text: "First item", options: { bullet: true, breakLine: true } },
{ text: "Second item", options: { bullet: true, breakLine: true } },
{ text: "Third item", options: { bullet: true } }
], { x: 0.5, y: 0.5, w: 8, h: 3 });
// ❌ WRONG: Never use unicode bullets
slide.addText("• First item", { ... }); // Creates double bullets
// Sub-items and numbered lists
{ text: "Sub-item", options: { bullet: true, indentLevel: 1 } }
{ text: "First", options: { bullet: { type: "number" }, breakLine: true } }
```
---
## Shapes
```javascript
slide.addShape(pres.shapes.RECTANGLE, {
x: 0.5, y: 0.8, w: 1.5, h: 3.0,
fill: { color: "FF0000" }, line: { color: "000000", width: 2 }
});
slide.addShape(pres.shapes.OVAL, { x: 4, y: 1, w: 2, h: 2, fill: { color: "0000FF" } });
slide.addShape(pres.shapes.LINE, {
x: 1, y: 3, w: 5, h: 0, line: { color: "FF0000", width: 3, dashType: "dash" }
});
// With transparency
slide.addShape(pres.shapes.RECTANGLE, {
x: 1, y: 1, w: 3, h: 2,
fill: { color: "0088CC", transparency: 50 }
});
// Rounded rectangle (rectRadius only works with ROUNDED_RECTANGLE, not RECTANGLE)
// ⚠️ Don't pair with rectangular accent overlays — they won't cover rounded corners. Use RECTANGLE instead.
slide.addShape(pres.shapes.ROUNDED_RECTANGLE, {
x: 1, y: 1, w: 3, h: 2,
fill: { color: "FFFFFF" }, rectRadius: 0.1
});
// With shadow
slide.addShape(pres.shapes.RECTANGLE, {
x: 1, y: 1, w: 3, h: 2,
fill: { color: "FFFFFF" },
shadow: { type: "outer", color: "000000", blur: 6, offset: 2, angle: 135, opacity: 0.15 }
});
```
Shadow options:
| Property | Type | Range | Notes |
|----------|------|-------|-------|
| `type` | string | `"outer"`, `"inner"` | |
| `color` | string | 6-char hex (e.g. `"000000"`) | No `#` prefix, no 8-char hex — see Common Pitfalls |
| `blur` | number | 0-100 pt | |
| `offset` | number | 0-200 pt | **Must be non-negative** — negative values corrupt the file |
| `angle` | number | 0-359 degrees | Direction the shadow falls (135 = bottom-right, 270 = upward) |
| `opacity` | number | 0.0-1.0 | Use this for transparency, never encode in color string |
To cast a shadow upward (e.g. on a footer bar), use `angle: 270` with a positive offset — do **not** use a negative offset.
**Note**: Gradient fills are not natively supported. Use a gradient image as a background instead.
---
## Images
### Image Sources
```javascript
// From file path
slide.addImage({ path: "images/chart.png", x: 1, y: 1, w: 5, h: 3 });
// From URL
slide.addImage({ path: "https://example.com/image.jpg", x: 1, y: 1, w: 5, h: 3 });
// From base64 (faster, no file I/O)
slide.addImage({ data: "image/png;base64,iVBORw0KGgo...", x: 1, y: 1, w: 5, h: 3 });
```
### Image Options
```javascript
slide.addImage({
path: "image.png",
x: 1, y: 1, w: 5, h: 3,
rotate: 45, // 0-359 degrees
rounding: true, // Circular crop
transparency: 50, // 0-100
flipH: true, // Horizontal flip
flipV: false, // Vertical flip
altText: "Description", // Accessibility
hyperlink: { url: "https://example.com" }
});
```
### Image Sizing Modes
```javascript
// Contain - fit inside, preserve ratio
{ sizing: { type: 'contain', w: 4, h: 3 } }
// Cover - fill area, preserve ratio (may crop)
{ sizing: { type: 'cover', w: 4, h: 3 } }
// Crop - cut specific portion
{ sizing: { type: 'crop', x: 0.5, y: 0.5, w: 2, h: 2 } }
```
### Calculate Dimensions (preserve aspect ratio)
```javascript
const origWidth = 1978, origHeight = 923, maxHeight = 3.0;
const calcWidth = maxHeight * (origWidth / origHeight);
const centerX = (10 - calcWidth) / 2;
slide.addImage({ path: "image.png", x: centerX, y: 1.2, w: calcWidth, h: maxHeight });
```
### Supported Formats
- **Standard**: PNG, JPG, GIF (animated GIFs work in Microsoft 365)
- **SVG**: Works in modern PowerPoint/Microsoft 365
---
## Icons
Use react-icons to generate SVG icons, then rasterize to PNG for universal compatibility.
### Setup
```javascript
const React = require("react");
const ReactDOMServer = require("react-dom/server");
const sharp = require("sharp");
const { FaCheckCircle, FaChartLine } = require("react-icons/fa");
function renderIconSvg(IconComponent, color = "#000000", size = 256) {
return ReactDOMServer.renderToStaticMarkup(
React.createElement(IconComponent, { color, size: String(size) })
);
}
async function iconToBase64Png(IconComponent, color, size = 256) {
const svg = renderIconSvg(IconComponent, color, size);
const pngBuffer = await sharp(Buffer.from(svg)).png().toBuffer();
return "image/png;base64," + pngBuffer.toString("base64");
}
```
### Add Icon to Slide
```javascript
const iconData = await iconToBase64Png(FaCheckCircle, "#4472C4", 256);
slide.addImage({
data: iconData,
x: 1, y: 1, w: 0.5, h: 0.5 // Size in inches
});
```
**Note**: Use size 256 or higher for crisp icons. The size parameter controls the rasterization resolution, not the display size on the slide (which is set by `w` and `h` in inches).
### Icon Libraries
Install: `npm install -g react-icons react react-dom sharp`
Popular icon sets in react-icons:
- `react-icons/fa` - Font Awesome
- `react-icons/md` - Material Design
- `react-icons/hi` - Heroicons
- `react-icons/bi` - Bootstrap Icons
---
## Slide Backgrounds
```javascript
// Solid color
slide.background = { color: "F1F1F1" };
// Color with transparency
slide.background = { color: "FF3399", transparency: 50 };
// Image from URL
slide.background = { path: "https://example.com/bg.jpg" };
// Image from base64
slide.background = { data: "image/png;base64,iVBORw0KGgo..." };
```
---
## Tables
```javascript
slide.addTable([
["Header 1", "Header 2"],
["Cell 1", "Cell 2"]
], {
x: 1, y: 1, w: 8, h: 2,
border: { pt: 1, color: "999999" }, fill: { color: "F1F1F1" }
});
// Advanced with merged cells
let tableData = [
[{ text: "Header", options: { fill: { color: "6699CC" }, color: "FFFFFF", bold: true } }, "Cell"],
[{ text: "Merged", options: { colspan: 2 } }]
];
slide.addTable(tableData, { x: 1, y: 3.5, w: 8, colW: [4, 4] });
```
---
## Charts
```javascript
// Bar chart
slide.addChart(pres.charts.BAR, [{
name: "Sales", labels: ["Q1", "Q2", "Q3", "Q4"], values: [4500, 5500, 6200, 7100]
}], {
x: 0.5, y: 0.6, w: 6, h: 3, barDir: 'col',
showTitle: true, title: 'Quarterly Sales'
});
// Line chart
slide.addChart(pres.charts.LINE, [{
name: "Temp", labels: ["Jan", "Feb", "Mar"], values: [32, 35, 42]
}], { x: 0.5, y: 4, w: 6, h: 3, lineSize: 3, lineSmooth: true });
// Pie chart
slide.addChart(pres.charts.PIE, [{
name: "Share", labels: ["A", "B", "Other"], values: [35, 45, 20]
}], { x: 7, y: 1, w: 5, h: 4, showPercent: true });
```
### Better-Looking Charts
Default charts look dated. Apply these options for a modern, clean appearance:
```javascript
slide.addChart(pres.charts.BAR, chartData, {
x: 0.5, y: 1, w: 9, h: 4, barDir: "col",
// Custom colors (match your presentation palette)
chartColors: ["0D9488", "14B8A6", "5EEAD4"],
// Clean background
chartArea: { fill: { color: "FFFFFF" }, roundedCorners: true },
// Muted axis labels
catAxisLabelColor: "64748B",
valAxisLabelColor: "64748B",
// Subtle grid (value axis only)
valGridLine: { color: "E2E8F0", size: 0.5 },
catGridLine: { style: "none" },
// Data labels on bars
showValue: true,
dataLabelPosition: "outEnd",
dataLabelColor: "1E293B",
// Hide legend for single series
showLegend: false,
});
```
**Key styling options:**
- `chartColors: [...]` - hex colors for series/segments
- `chartArea: { fill, border, roundedCorners }` - chart background
- `catGridLine/valGridLine: { color, style, size }` - grid lines (`style: "none"` to hide)
- `lineSmooth: true` - curved lines (line charts)
- `legendPos: "r"` - legend position: "b", "t", "l", "r", "tr"
---
## Slide Masters
```javascript
pres.defineSlideMaster({
title: 'TITLE_SLIDE', background: { color: '283A5E' },
objects: [{
placeholder: { options: { name: 'title', type: 'title', x: 1, y: 2, w: 8, h: 2 } }
}]
});
let titleSlide = pres.addSlide({ masterName: "TITLE_SLIDE" });
titleSlide.addText("My Title", { placeholder: "title" });
```
---
## Common Pitfalls
⚠️ These issues cause file corruption, visual bugs, or broken output. Avoid them.
1. **NEVER use "#" with hex colors** - causes file corruption
```javascript
color: "FF0000" // ✅ CORRECT
color: "#FF0000" // ❌ WRONG
```
2. **NEVER encode opacity in hex color strings** - 8-char colors (e.g., `"00000020"`) corrupt the file. Use the `opacity` property instead.
```javascript
shadow: { type: "outer", blur: 6, offset: 2, color: "00000020" } // ❌ CORRUPTS FILE
shadow: { type: "outer", blur: 6, offset: 2, color: "000000", opacity: 0.12 } // ✅ CORRECT
```
3. **Use `bullet: true`** - NEVER unicode symbols like "•" (creates double bullets)
4. **Use `breakLine: true`** between array items or text runs together
5. **Avoid `lineSpacing` with bullets** - causes excessive gaps; use `paraSpaceAfter` instead
6. **Each presentation needs fresh instance** - don't reuse `pptxgen()` objects
7. **NEVER reuse option objects across calls** - PptxGenJS mutates objects in-place (e.g. converting shadow values to EMU). Sharing one object between multiple calls corrupts the second shape.
```javascript
const shadow = { type: "outer", blur: 6, offset: 2, color: "000000", opacity: 0.15 };
slide.addShape(pres.shapes.RECTANGLE, { shadow, ... }); // ❌ second call gets already-converted values
slide.addShape(pres.shapes.RECTANGLE, { shadow, ... });
const makeShadow = () => ({ type: "outer", blur: 6, offset: 2, color: "000000", opacity: 0.15 });
slide.addShape(pres.shapes.RECTANGLE, { shadow: makeShadow(), ... }); // ✅ fresh object each time
slide.addShape(pres.shapes.RECTANGLE, { shadow: makeShadow(), ... });
```
8. **Don't use `ROUNDED_RECTANGLE` with accent borders** - rectangular overlay bars won't cover rounded corners. Use `RECTANGLE` instead.
```javascript
// ❌ WRONG: Accent bar doesn't cover rounded corners
slide.addShape(pres.shapes.ROUNDED_RECTANGLE, { x: 1, y: 1, w: 3, h: 1.5, fill: { color: "FFFFFF" } });
slide.addShape(pres.shapes.RECTANGLE, { x: 1, y: 1, w: 0.08, h: 1.5, fill: { color: "0891B2" } });
// ✅ CORRECT: Use RECTANGLE for clean alignment
slide.addShape(pres.shapes.RECTANGLE, { x: 1, y: 1, w: 3, h: 1.5, fill: { color: "FFFFFF" } });
slide.addShape(pres.shapes.RECTANGLE, { x: 1, y: 1, w: 0.08, h: 1.5, fill: { color: "0891B2" } });
```
---
## Quick Reference
- **Shapes**: RECTANGLE, OVAL, LINE, ROUNDED_RECTANGLE
- **Charts**: BAR, LINE, PIE, DOUGHNUT, SCATTER, BUBBLE, RADAR
- **Layouts**: LAYOUT_16x9 (10"×5.625"), LAYOUT_16x10, LAYOUT_4x3, LAYOUT_WIDE
- **Alignment**: "left", "center", "right"
- **Chart data labels**: "outEnd", "inEnd", "center"
skills/pptx/scripts/add_slide.py
+312 -140
View File
@@ -1,51 +1,41 @@
"""Add a new slide to an unpacked PPTX directory. """Add a slide to a PPTX: duplicate an existing slide or instantiate a layout.
Usage: python add_slide.py <unpacked_dir> <source> Does all of the package bookkeeping, so the deck stays valid:
- writes the new ppt/slides/slideN.xml (and its .rels, minus any
notesSlide reference, so the source's speaker notes aren't shared)
- registers it in [Content_Types].xml
- adds a slide relationship with a fresh rId to presentation.xml.rels
- inserts <p:sldId id="..." r:id="..."/> with a fresh id into
<p:sldIdLst> — at the end, or after --after SLIDE
The source can be: Works on an unpacked directory (during an editing session) or directly on a
- A slide file (e.g., slide2.xml) - duplicates the slide .pptx/.potx file (extracted to a temp dir, then rezipped atomically; the
- A layout file (e.g., slideLayout2.xml) - creates from layout temp dir is discarded, so unpack the output if you still need to edit the
new slide's content).
Examples: Usage:
python add_slide.py unpacked/ slide2.xml python add_slide.py unpacked/ slide2.xml # duplicate slide2
# Duplicates slide2, creates slide5.xml python add_slide.py unpacked/ slideLayout3.xml # new slide from a layout
python add_slide.py unpacked/ slide2.xml --after slide2.xml
python add_slide.py deck.pptx slide2.xml # rewrite deck.pptx in place
python add_slide.py deck.pptx slide2.xml -o out.pptx
python add_slide.py unpacked/ slideLayout2.xml A duplicated slide still holds the source's content: edit ppt/slides/slideN.xml
# Creates slide5.xml from slideLayout2.xml (printed on success) to change it. To list layouts: ls <dir>/ppt/slideLayouts/
To see available layouts: ls unpacked/ppt/slideLayouts/
Prints the <p:sldId> element to add to presentation.xml.
""" """
import argparse
import re import re
import shutil import shutil
import sys import sys
from typing import NoReturn
import tempfile
import zipfile
from pathlib import Path from pathlib import Path
from office.helpers import rezip, safe_extract
def get_next_slide_number(slides_dir: Path) -> int: MINIMAL_SLIDE_XML = '''<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
existing = [int(m.group(1)) for f in slides_dir.glob("slide*.xml")
if (m := re.match(r"slide(\d+)\.xml", f.name))]
return max(existing) + 1 if existing else 1
def create_slide_from_layout(unpacked_dir: Path, layout_file: str) -> None:
slides_dir = unpacked_dir / "ppt" / "slides"
rels_dir = slides_dir / "_rels"
layouts_dir = unpacked_dir / "ppt" / "slideLayouts"
layout_path = layouts_dir / layout_file
if not layout_path.exists():
print(f"Error: {layout_path} not found", file=sys.stderr)
sys.exit(1)
next_num = get_next_slide_number(slides_dir)
dest = f"slide{next_num}.xml"
dest_slide = slides_dir / dest
dest_rels = rels_dir / f"{dest}.rels"
slide_xml = '''<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<p:sld xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:p="http://schemas.openxmlformats.org/presentationml/2006/main"> <p:sld xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:p="http://schemas.openxmlformats.org/presentationml/2006/main">
<p:cSld> <p:cSld>
<p:spTree> <p:spTree>
@@ -68,98 +58,25 @@ def create_slide_from_layout(unpacked_dir: Path, layout_file: str) -> None:
<a:masterClrMapping/> <a:masterClrMapping/>
</p:clrMapOvr> </p:clrMapOvr>
</p:sld>''' </p:sld>'''
dest_slide.write_text(slide_xml, encoding="utf-8")
rels_dir.mkdir(exist_ok=True) SHARED_PART_TYPES = ("chart", "diagramData", "oleObject", "package")
rels_xml = f'''<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/slideLayout" Target="../slideLayouts/{layout_file}"/>
</Relationships>'''
dest_rels.write_text(rels_xml, encoding="utf-8")
_add_to_content_types(unpacked_dir, dest) NOTES_SLIDE_TYPE_RE = re.compile(r"""Type=["'][^"']*/relationships/notesSlide["']""")
RELATIONSHIP_RE = re.compile(r"<Relationship\b[^>]*?(?:/>|>.*?</Relationship\s*>)", re.DOTALL)
rid = _add_to_presentation_rels(unpacked_dir, dest) SLIDE_ID_MIN = 256
SLIDE_ID_MAX = 2147483647
next_slide_id = _get_next_slide_id(unpacked_dir)
print(f"Created {dest} from {layout_file}")
print(f'Add to presentation.xml <p:sldIdLst>: <p:sldId id="{next_slide_id}" r:id="{rid}"/>')
def duplicate_slide(unpacked_dir: Path, source: str) -> None: def _die(msg: str) -> NoReturn:
slides_dir = unpacked_dir / "ppt" / "slides" print(f"Error: {msg}", file=sys.stderr)
rels_dir = slides_dir / "_rels" sys.exit(1)
source_slide = slides_dir / source
if not source_slide.exists():
print(f"Error: {source_slide} not found", file=sys.stderr)
sys.exit(1)
next_num = get_next_slide_number(slides_dir)
dest = f"slide{next_num}.xml"
dest_slide = slides_dir / dest
source_rels = rels_dir / f"{source}.rels"
dest_rels = rels_dir / f"{dest}.rels"
shutil.copy2(source_slide, dest_slide)
if source_rels.exists():
shutil.copy2(source_rels, dest_rels)
rels_content = dest_rels.read_text(encoding="utf-8")
rels_content = re.sub(
r'\s*<Relationship[^>]*Type="[^"]*notesSlide"[^>]*/>\s*',
"\n",
rels_content,
)
dest_rels.write_text(rels_content, encoding="utf-8")
_add_to_content_types(unpacked_dir, dest)
rid = _add_to_presentation_rels(unpacked_dir, dest)
next_slide_id = _get_next_slide_id(unpacked_dir)
print(f"Created {dest} from {source}")
print(f'Add to presentation.xml <p:sldIdLst>: <p:sldId id="{next_slide_id}" r:id="{rid}"/>')
def _add_to_content_types(unpacked_dir: Path, dest: str) -> None: def get_next_slide_number(slides_dir: Path) -> int:
content_types_path = unpacked_dir / "[Content_Types].xml" existing = [int(m.group(1)) for f in slides_dir.glob("slide*.xml")
content_types = content_types_path.read_text(encoding="utf-8") if (m := re.match(r"slide(\d+)\.xml", f.name))]
return max(existing) + 1 if existing else 1
new_override = f'<Override PartName="/ppt/slides/{dest}" ContentType="application/vnd.openxmlformats-officedocument.presentationml.slide+xml"/>'
if f"/ppt/slides/{dest}" not in content_types:
content_types = content_types.replace("</Types>", f" {new_override}\n</Types>")
content_types_path.write_text(content_types, encoding="utf-8")
def _add_to_presentation_rels(unpacked_dir: Path, dest: str) -> str:
pres_rels_path = unpacked_dir / "ppt" / "_rels" / "presentation.xml.rels"
pres_rels = pres_rels_path.read_text(encoding="utf-8")
rids = [int(m) for m in re.findall(r'Id="rId(\d+)"', pres_rels)]
next_rid = max(rids) + 1 if rids else 1
rid = f"rId{next_rid}"
new_rel = f'<Relationship Id="{rid}" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/slide" Target="slides/{dest}"/>'
if f"slides/{dest}" not in pres_rels:
pres_rels = pres_rels.replace("</Relationships>", f" {new_rel}\n</Relationships>")
pres_rels_path.write_text(pres_rels, encoding="utf-8")
return rid
def _get_next_slide_id(unpacked_dir: Path) -> int:
pres_path = unpacked_dir / "ppt" / "presentation.xml"
pres_content = pres_path.read_text(encoding="utf-8")
slide_ids = [int(m) for m in re.findall(r'<p:sldId[^>]*id="(\d+)"', pres_content)]
return max(slide_ids) + 1 if slide_ids else 256
def parse_source(source: str) -> tuple[str, str | None]: def parse_source(source: str) -> tuple[str, str | None]:
@@ -169,27 +86,282 @@ def parse_source(source: str) -> tuple[str, str | None]:
return ("slide", None) return ("slide", None)
if __name__ == "__main__": def create_slide_from_layout(unpacked_dir: Path, layout_file: str, after: str | None = None) -> str:
if len(sys.argv) != 3: slides_dir = unpacked_dir / "ppt" / "slides"
print("Usage: python add_slide.py <unpacked_dir> <source>", file=sys.stderr) rels_dir = slides_dir / "_rels"
print("", file=sys.stderr) layout_path = unpacked_dir / "ppt" / "slideLayouts" / layout_file
print("Source can be:", file=sys.stderr)
print(" slide2.xml - duplicate an existing slide", file=sys.stderr)
print(" slideLayout2.xml - create from a layout template", file=sys.stderr)
print("", file=sys.stderr)
print("To see available layouts: ls <unpacked_dir>/ppt/slideLayouts/", file=sys.stderr)
sys.exit(1)
unpacked_dir = Path(sys.argv[1]) if not layout_path.exists():
source = sys.argv[2] _die(f"{layout_path} not found")
if not unpacked_dir.exists(): next_num = get_next_slide_number(slides_dir)
print(f"Error: {unpacked_dir} not found", file=sys.stderr) dest = f"slide{next_num}.xml"
sys.exit(1) after_rid = _precheck_registration(unpacked_dir, after, dest)
slides_dir.mkdir(parents=True, exist_ok=True)
source_type, layout_file = parse_source(source) (slides_dir / dest).write_text(MINIMAL_SLIDE_XML, encoding="utf-8")
if source_type == "layout" and layout_file is not None: rels_dir.mkdir(exist_ok=True)
create_slide_from_layout(unpacked_dir, layout_file) rels_xml = f'''<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/slideLayout" Target="../slideLayouts/{layout_file}"/>
</Relationships>'''
(rels_dir / f"{dest}.rels").write_text(rels_xml, encoding="utf-8")
_register_slide(unpacked_dir, dest, layout_file, after_rid)
return dest
def duplicate_slide(unpacked_dir: Path, source: str, after: str | None = None) -> str:
slides_dir = unpacked_dir / "ppt" / "slides"
rels_dir = slides_dir / "_rels"
source_slide = slides_dir / source
if not source_slide.exists():
_die(f"{source_slide} not found")
next_num = get_next_slide_number(slides_dir)
dest = f"slide{next_num}.xml"
after_rid = _precheck_registration(unpacked_dir, after, dest)
shutil.copy2(source_slide, slides_dir / dest)
source_rels = rels_dir / f"{source}.rels"
shared_parts: list[str] = []
if source_rels.exists():
dest_rels = rels_dir / f"{dest}.rels"
shutil.copy2(source_rels, dest_rels)
rels_content = dest_rels.read_text(encoding="utf-8")
rels_content = RELATIONSHIP_RE.sub(
lambda m: "" if NOTES_SLIDE_TYPE_RE.search(m.group(0)) else m.group(0),
rels_content,
)
dest_rels.write_text(rels_content, encoding="utf-8")
shared_parts = sorted({
t for t in re.findall(r'Type="[^"]*/relationships/(\w+)"', rels_content)
if t in SHARED_PART_TYPES
})
_register_slide(unpacked_dir, dest, source, after_rid)
if shared_parts:
print(
f"Note: {dest} shares its {', '.join(shared_parts)} part(s) with {source} "
f"(they are referenced, not copied) — editing those parts changes both slides"
)
return dest
def _precheck_registration(unpacked_dir: Path, after: str | None, dest: str) -> str | None:
pres_path = unpacked_dir / "ppt" / "presentation.xml"
if not pres_path.exists():
_die(f"{pres_path} not found — is this an unpacked PPTX?")
xml = pres_path.read_text(encoding="utf-8")
has_slot = (
"</p:sldIdLst>" in xml
or re.search(r"<p:sldIdLst\s*/>", xml)
or "</p:sldMasterIdLst>" in xml
)
if not has_slot:
_die("presentation.xml has no <p:sldIdLst> (or <p:sldMasterIdLst> to anchor a new one)")
stale = []
content_types = unpacked_dir / "[Content_Types].xml"
if content_types.exists() and f'PartName="/ppt/slides/{dest}"' in content_types.read_text(encoding="utf-8"):
stale.append("[Content_Types].xml")
pres_rels = unpacked_dir / "ppt" / "_rels" / "presentation.xml.rels"
if pres_rels.exists() and _find_slide_relationship(
pres_rels.read_text(encoding="utf-8"), dest
):
stale.append("presentation.xml.rels")
if stale:
_die(
f"{dest} is still registered in {' and '.join(stale)} but absent from ppt/slides/ — "
f"run clean.py first"
)
if not after:
return None
after_rid = _rid_for_slide(unpacked_dir, after)
if not re.search(rf'<p:sldId\b[^>]*r:id="{re.escape(after_rid)}"[^>]*>', xml):
_die(f"{after} ({after_rid}) is not listed in <p:sldIdLst>")
return after_rid
def _register_slide(unpacked_dir: Path, dest: str, source_desc: str, after_rid: str | None) -> None:
_add_to_content_types(unpacked_dir, dest)
rid = _add_to_presentation_rels(unpacked_dir, dest)
slide_id = _get_next_slide_id(unpacked_dir)
pos, total = _insert_into_sld_id_lst(unpacked_dir, slide_id, rid, after_rid)
print(f"Created ppt/slides/{dest} from {source_desc}")
print(
f'Inserted <p:sldId id="{slide_id}" r:id="{rid}"/> into <p:sldIdLst> '
f"at position {pos} of {total}"
)
def _add_to_content_types(unpacked_dir: Path, dest: str) -> None:
content_types_path = unpacked_dir / "[Content_Types].xml"
content_types = content_types_path.read_text(encoding="utf-8")
new_override = f'<Override PartName="/ppt/slides/{dest}" ContentType="application/vnd.openxmlformats-officedocument.presentationml.slide+xml"/>'
if f'PartName="/ppt/slides/{dest}"' not in content_types:
content_types = content_types.replace("</Types>", f" {new_override}\n</Types>")
content_types_path.write_text(content_types, encoding="utf-8")
def _add_to_presentation_rels(unpacked_dir: Path, dest: str) -> str:
pres_rels_path = unpacked_dir / "ppt" / "_rels" / "presentation.xml.rels"
pres_rels = pres_rels_path.read_text(encoding="utf-8")
existing = _find_slide_relationship(pres_rels, dest)
if existing:
return existing
pres_xml = (unpacked_dir / "ppt" / "presentation.xml").read_text(encoding="utf-8")
used = {int(n) for n in re.findall(r'\bId="rId(\d+)"', pres_rels)}
used |= {int(n) for n in re.findall(r'\br:id="rId(\d+)"', pres_xml)}
rid = f"rId{max(used) + 1 if used else 1}"
new_rel = f'<Relationship Id="{rid}" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/slide" Target="slides/{dest}"/>'
pres_rels = pres_rels.replace("</Relationships>", f" {new_rel}\n</Relationships>")
pres_rels_path.write_text(pres_rels, encoding="utf-8")
return rid
def _find_slide_relationship(pres_rels: str, slide_name: str) -> str | None:
for m in re.finditer(r"<Relationship\b[^>]*>", pres_rels):
element = m.group(0)
if re.search(rf'Target="(?:/ppt/)?slides/{re.escape(slide_name)}"', element):
id_match = re.search(r'\bId="([^"]+)"', element)
if id_match:
return id_match.group(1)
return None
def _get_next_slide_id(unpacked_dir: Path) -> int:
pres_content = (unpacked_dir / "ppt" / "presentation.xml").read_text(encoding="utf-8")
used = {int(m) for m in re.findall(r'<p:sldId[^>]*\bid="(\d+)"', pres_content)}
candidate = max((i for i in used if i >= SLIDE_ID_MIN), default=SLIDE_ID_MIN - 1) + 1
if candidate <= SLIDE_ID_MAX and candidate not in used:
return candidate
for i in range(SLIDE_ID_MIN, SLIDE_ID_MAX + 1):
if i not in used:
return i
_die("no slide id available in [256, 2147483647] — the deck is full")
def _insert_into_sld_id_lst(
unpacked_dir: Path, slide_id: int, rid: str, after_rid: str | None = None
) -> tuple[int, int]:
pres_path = unpacked_dir / "ppt" / "presentation.xml"
xml = pres_path.read_text(encoding="utf-8")
entry = f'<p:sldId id="{slide_id}" r:id="{rid}"/>'
if f'r:id="{rid}"' in xml:
_die(f"presentation.xml already references {rid}; refusing to add a duplicate")
if after_rid:
open_tag = re.search(rf'<p:sldId\b[^>]*r:id="{re.escape(after_rid)}"[^>]*>', xml)
if not open_tag:
_die(f"{after_rid} is not listed in <p:sldIdLst>")
end = open_tag.end()
if not open_tag.group(0).endswith("/>"):
close = xml.find("</p:sldId>", end)
if close == -1:
_die(f"unclosed <p:sldId> for {after_rid} in presentation.xml")
end = close + len("</p:sldId>")
xml = xml[:end] + entry + xml[end:]
elif "</p:sldIdLst>" in xml:
xml = xml.replace("</p:sldIdLst>", f"{entry}</p:sldIdLst>", 1)
elif re.search(r"<p:sldIdLst\s*/>", xml):
xml = re.sub(r"<p:sldIdLst\s*/>", f"<p:sldIdLst>{entry}</p:sldIdLst>", xml, count=1)
elif "</p:sldMasterIdLst>" in xml:
xml = xml.replace(
"</p:sldMasterIdLst>", f"</p:sldMasterIdLst><p:sldIdLst>{entry}</p:sldIdLst>", 1
)
else: else:
duplicate_slide(unpacked_dir, source) _die("presentation.xml has no <p:sldIdLst> (or <p:sldMasterIdLst> to anchor a new one)")
pres_path.write_text(xml, encoding="utf-8")
lst = re.search(r"<p:sldIdLst>(.*)</p:sldIdLst>", xml, re.DOTALL)
entries = re.findall(r"<p:sldId\b[^>]*>", lst.group(1)) if lst else []
position = next(
(i for i, e in enumerate(entries, 1) if f'r:id="{rid}"' in e), len(entries)
)
return position, len(entries)
def _rid_for_slide(unpacked_dir: Path, slide_name: str) -> str:
pres_rels_path = unpacked_dir / "ppt" / "_rels" / "presentation.xml.rels"
rid = _find_slide_relationship(pres_rels_path.read_text(encoding="utf-8"), slide_name)
if not rid:
_die(f"{slide_name} has no relationship in presentation.xml.rels")
return rid
def add_slide(unpacked_dir: Path, source: str, after: str | None = None) -> str:
source_type, layout_file = parse_source(source)
if source_type == "layout" and layout_file is not None:
return create_slide_from_layout(unpacked_dir, layout_file, after)
return duplicate_slide(unpacked_dir, source, after)
def add_slide_to_package(
package: Path, source: str, after: str | None = None, output: Path | None = None
) -> str:
out = output or package
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
with zipfile.ZipFile(package) as zf:
safe_extract(zf, tmp_path)
dest = add_slide(tmp_path, source, after)
rezip(tmp_path, out)
print(f"Wrote {out} — the new slide is ppt/slides/{dest} inside it (unpack to edit its content)")
return dest
def main() -> None:
parser = argparse.ArgumentParser(
description="Add a slide to a PPTX: duplicate a slide or instantiate a layout. "
"Registers content types, relationships, and <p:sldIdLst>."
)
parser.add_argument("target", help="Unpacked PPTX directory OR a .pptx/.potx file")
parser.add_argument(
"source",
help="slideN.xml to duplicate, or slideLayoutN.xml to create from a layout "
"(list layouts with: ls <dir>/ppt/slideLayouts/)",
)
parser.add_argument(
"--after",
metavar="SLIDE",
help="insert after this slide, e.g. slide2.xml (default: append at the end)",
)
parser.add_argument(
"-o",
"--output",
help="output file (only with a .pptx/.potx target; default: rewrite the input in place)",
)
args = parser.parse_args()
target = Path(args.target)
if target.is_dir():
if args.output:
parser.error("--output is only valid for .pptx/.potx input; a directory is modified in place")
add_slide(target, args.source, args.after)
elif target.is_file() and target.suffix.lower() in (".pptx", ".potx"):
try:
add_slide_to_package(target, args.source, args.after, Path(args.output) if args.output else None)
except (OSError, ValueError, zipfile.BadZipFile) as e:
_die(str(e))
else:
_die(f"{target} is neither a directory nor a .pptx/.potx file")
if __name__ == "__main__":
main()
skills/pptx/scripts/clean.py
+80 -57
View File
@@ -15,13 +15,30 @@ This script removes:
- Content-Type overrides for deleted files - Content-Type overrides for deleted files
""" """
import posixpath
import re
import sys import sys
from pathlib import Path from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
from office.helpers import SLIDE_REL_TYPE, opc_target, rels_source_part
import re
def _slide_rids(pres_rels_path: Path, unpacked_dir: Path) -> dict[str, str]:
source_part = rels_source_part(pres_rels_path, unpacked_dir)
rels_dom = defusedxml.minidom.parse(str(pres_rels_path))
rids: dict[str, str] = {}
for rel in rels_dom.getElementsByTagName("Relationship"):
if rel.getAttribute("Type") != SLIDE_REL_TYPE:
continue
part = opc_target(
rel.getAttribute("Target"), source_part, rel.getAttribute("TargetMode")
)
if part is not None:
rids[rel.getAttribute("Id")] = part
return rids
def get_slides_in_sldidlst(unpacked_dir: Path) -> set[str]: def get_slides_in_sldidlst(unpacked_dir: Path) -> set[str]:
@@ -31,19 +48,20 @@ def get_slides_in_sldidlst(unpacked_dir: Path) -> set[str]:
if not pres_path.exists() or not pres_rels_path.exists(): if not pres_path.exists() or not pres_rels_path.exists():
return set() return set()
rels_dom = defusedxml.minidom.parse(str(pres_rels_path)) rid_to_slide = _slide_rids(pres_rels_path, unpacked_dir)
rid_to_slide = {}
for rel in rels_dom.getElementsByTagName("Relationship"):
rid = rel.getAttribute("Id")
target = rel.getAttribute("Target")
rel_type = rel.getAttribute("Type")
if "slide" in rel_type and target.startswith("slides/"):
rid_to_slide[rid] = target.replace("slides/", "")
pres_content = pres_path.read_text(encoding="utf-8") pres_content = pres_path.read_text(encoding="utf-8")
referenced_rids = set(re.findall(r'<p:sldId[^>]*r:id="([^"]+)"', pres_content)) referenced_rids = set(re.findall(r'<p:sldId[^>]*r:id="([^"]+)"', pres_content))
return {rid_to_slide[rid] for rid in referenced_rids if rid in rid_to_slide} return {
posixpath.basename(rid_to_slide[rid])
for rid in referenced_rids
if rid in rid_to_slide
}
class RefusedToClean(Exception):
"""The package does not look the way a readable package should."""
def remove_orphaned_slides(unpacked_dir: Path) -> list[str]: def remove_orphaned_slides(unpacked_dir: Path) -> list[str]:
@@ -55,9 +73,25 @@ def remove_orphaned_slides(unpacked_dir: Path) -> list[str]:
return [] return []
referenced_slides = get_slides_in_sldidlst(unpacked_dir) referenced_slides = get_slides_in_sldidlst(unpacked_dir)
on_disk = sorted(slides_dir.glob("slide*.xml"))
if on_disk and not any(s.name in referenced_slides for s in on_disk):
listed = re.findall(
r'<p:sldId[^>]*r:id="([^"]+)"',
(unpacked_dir / "ppt" / "presentation.xml").read_text(encoding="utf-8")
if (unpacked_dir / "ppt" / "presentation.xml").exists()
else "",
)
if listed:
raise RefusedToClean(
f"<p:sldIdLst> lists {len(listed)} slide(s) and none of the "
f"{len(on_disk)} slide(s) on disk match any of them. Refusing to "
f"delete them all — this is a parse failure, not an empty deck."
)
removed = [] removed = []
for slide_file in slides_dir.glob("slide*.xml"): for slide_file in on_disk:
if slide_file.name not in referenced_slides: if slide_file.name not in referenced_slides:
rel_path = slide_file.relative_to(unpacked_dir) rel_path = slide_file.relative_to(unpacked_dir)
slide_file.unlink() slide_file.unlink()
@@ -70,16 +104,21 @@ def remove_orphaned_slides(unpacked_dir: Path) -> list[str]:
if removed and pres_rels_path.exists(): if removed and pres_rels_path.exists():
rels_dom = defusedxml.minidom.parse(str(pres_rels_path)) rels_dom = defusedxml.minidom.parse(str(pres_rels_path))
source_part = rels_source_part(pres_rels_path, unpacked_dir)
changed = False changed = False
for rel in list(rels_dom.getElementsByTagName("Relationship")): for rel in list(rels_dom.getElementsByTagName("Relationship")):
target = rel.getAttribute("Target") if rel.getAttribute("Type") != SLIDE_REL_TYPE:
if target.startswith("slides/"): continue
slide_name = target.replace("slides/", "") part = opc_target(
if slide_name not in referenced_slides: rel.getAttribute("Target"), source_part, rel.getAttribute("TargetMode")
if rel.parentNode: )
rel.parentNode.removeChild(rel) if part is None:
changed = True continue
if posixpath.basename(part) not in referenced_slides:
if rel.parentNode:
rel.parentNode.removeChild(rel)
changed = True
if changed: if changed:
with open(pres_rels_path, "wb") as f: with open(pres_rels_path, "wb") as f:
@@ -103,24 +142,18 @@ def remove_trash_directory(unpacked_dir: Path) -> list[str]:
return removed return removed
def get_slide_referenced_files(unpacked_dir: Path) -> set: def _referenced_by(rels_files, unpacked_dir: Path) -> set:
referenced = set() referenced = set()
slides_rels_dir = unpacked_dir / "ppt" / "slides" / "_rels"
if not slides_rels_dir.exists(): for rels_file in rels_files:
return referenced source_part = rels_source_part(rels_file, unpacked_dir)
for rels_file in slides_rels_dir.glob("*.rels"):
dom = defusedxml.minidom.parse(str(rels_file)) dom = defusedxml.minidom.parse(str(rels_file))
for rel in dom.getElementsByTagName("Relationship"): for rel in dom.getElementsByTagName("Relationship"):
target = rel.getAttribute("Target") part = opc_target(
if not target: rel.getAttribute("Target"), source_part, rel.getAttribute("TargetMode")
continue )
target_path = (rels_file.parent.parent / target).resolve() if part is not None:
try: referenced.add(Path(part))
referenced.add(target_path.relative_to(unpacked_dir.resolve()))
except ValueError:
pass
return referenced return referenced
@@ -128,7 +161,6 @@ def get_slide_referenced_files(unpacked_dir: Path) -> set:
def remove_orphaned_rels_files(unpacked_dir: Path) -> list[str]: def remove_orphaned_rels_files(unpacked_dir: Path) -> list[str]:
resource_dirs = ["charts", "diagrams", "drawings"] resource_dirs = ["charts", "diagrams", "drawings"]
removed = [] removed = []
slide_referenced = get_slide_referenced_files(unpacked_dir)
for dir_name in resource_dirs: for dir_name in resource_dirs:
rels_dir = unpacked_dir / "ppt" / dir_name / "_rels" rels_dir = unpacked_dir / "ppt" / dir_name / "_rels"
@@ -137,35 +169,15 @@ def remove_orphaned_rels_files(unpacked_dir: Path) -> list[str]:
for rels_file in rels_dir.glob("*.rels"): for rels_file in rels_dir.glob("*.rels"):
resource_file = rels_dir.parent / rels_file.name.replace(".rels", "") resource_file = rels_dir.parent / rels_file.name.replace(".rels", "")
try: if not resource_file.exists():
resource_rel_path = resource_file.resolve().relative_to(unpacked_dir.resolve())
except ValueError:
continue
if not resource_file.exists() or resource_rel_path not in slide_referenced:
rels_file.unlink() rels_file.unlink()
rel_path = rels_file.relative_to(unpacked_dir) removed.append(str(rels_file.relative_to(unpacked_dir)))
removed.append(str(rel_path))
return removed return removed
def get_referenced_files(unpacked_dir: Path) -> set: def get_referenced_files(unpacked_dir: Path) -> set:
referenced = set() return _referenced_by(sorted(unpacked_dir.rglob("*.rels")), unpacked_dir)
for rels_file in unpacked_dir.rglob("*.rels"):
dom = defusedxml.minidom.parse(str(rels_file))
for rel in dom.getElementsByTagName("Relationship"):
target = rel.getAttribute("Target")
if not target:
continue
target_path = (rels_file.parent.parent / target).resolve()
try:
referenced.add(target_path.relative_to(unpacked_dir.resolve()))
except ValueError:
pass
return referenced
def remove_orphaned_files(unpacked_dir: Path, referenced: set) -> list[str]: def remove_orphaned_files(unpacked_dir: Path, referenced: set) -> list[str]:
@@ -241,6 +253,12 @@ def update_content_types(unpacked_dir: Path, removed_files: list[str]) -> None:
def clean_unused_files(unpacked_dir: Path) -> list[str]: def clean_unused_files(unpacked_dir: Path) -> list[str]:
all_removed = [] all_removed = []
if list(unpacked_dir.rglob("*.rels")) and not get_referenced_files(unpacked_dir):
raise RefusedToClean(
"no relationship in this package names a part we can resolve. "
"Refusing to treat every file as unreferenced."
)
slides_removed = remove_orphaned_slides(unpacked_dir) slides_removed = remove_orphaned_slides(unpacked_dir)
all_removed.extend(slides_removed) all_removed.extend(slides_removed)
@@ -276,7 +294,12 @@ if __name__ == "__main__":
print(f"Error: {unpacked_dir} not found", file=sys.stderr) print(f"Error: {unpacked_dir} not found", file=sys.stderr)
sys.exit(1) sys.exit(1)
removed = clean_unused_files(unpacked_dir) try:
removed = clean_unused_files(unpacked_dir)
except (RefusedToClean, ValueError) as e:
print(f"Error: {e}", file=sys.stderr)
print("Nothing was deleted.", file=sys.stderr)
sys.exit(1)
if removed: if removed:
print(f"Removed {len(removed)} unreferenced files:") print(f"Removed {len(removed)} unreferenced files:")
skills/pptx/scripts/office/helpers/__init__.py
+111
View File
@@ -0,0 +1,111 @@
import os
import posixpath
import re
import stat
import tempfile
import urllib.parse
import zipfile
from pathlib import Path
OOXML_FAMILY = {
".docx": "docx",
".dotx": "docx",
".pptx": "pptx",
".potx": "pptx",
".xlsx": "xlsx",
".xltx": "xlsx",
}
_SCHEME_RE = re.compile(r"^[A-Za-z][A-Za-z0-9+.\-]*:")
SLIDE_REL_TYPE = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/slide"
def opc_target(target: str, source_part: str, target_mode: str = "") -> str | None:
if not target:
return None
if target_mode.lower() == "external":
return None
if _SCHEME_RE.match(target):
return None
target = urllib.parse.unquote(target)
if "\\" in target:
raise ValueError(f"relationship target is not a POSIX part name: {target!r}")
if target.startswith("/"):
joined = target.lstrip("/")
else:
joined = posixpath.join(posixpath.dirname(source_part), target)
parts: list[str] = []
for segment in posixpath.normpath(joined).split("/"):
if segment in ("", "."):
continue
if segment == "..":
if not parts:
raise ValueError(f"relationship target escapes the package: {target!r}")
parts.pop()
else:
parts.append(segment)
if not parts:
raise ValueError(f"relationship target resolves to nothing: {target!r}")
return "/".join(parts)
def rels_source_part(rels_file: Path, unpacked_dir: Path) -> str:
owner_dir = rels_file.parent.parent.relative_to(unpacked_dir)
return posixpath.join(owner_dir.as_posix(), rels_file.name[: -len(".rels")]).lstrip("./")
def part_text(data: bytes) -> str:
return data.decode("utf-8", "surrogateescape")
XML_SPACE = " \t\r\n"
def rendered_text(text: str, preserve: bool) -> str:
return text if preserve else text.strip(XML_SPACE)
def safe_extract(zf: zipfile.ZipFile, dest: Path) -> None:
dest = dest.resolve()
for m in zf.infolist():
if stat.S_ISLNK(m.external_attr >> 16):
raise ValueError(f"symlink archive entry not allowed: {m.filename!r}")
target = (dest / m.filename).resolve()
if not target.is_relative_to(dest):
raise ValueError(f"unsafe archive entry: {m.filename!r}")
zf.extract(m, dest)
def rezip(src_dir: Path, out_path: Path) -> None:
files = sorted(p for p in src_dir.rglob("*") if p.is_file())
ct = src_dir / "[Content_Types].xml"
fd, tmp_name = tempfile.mkstemp(
prefix=out_path.name + ".", suffix=".tmp", dir=out_path.parent
)
tmp_out = Path(tmp_name)
try:
with os.fdopen(fd, "wb") as fh:
with zipfile.ZipFile(fh, "w", zipfile.ZIP_DEFLATED) as zf:
if ct.exists():
zf.write(ct, ct.relative_to(src_dir), compress_type=zipfile.ZIP_STORED)
for f in files:
if f == ct:
continue
zf.write(f, f.relative_to(src_dir))
if out_path.exists():
mode = out_path.stat().st_mode & 0o777
else:
umask = os.umask(0)
os.umask(umask)
mode = 0o666 & ~umask
os.chmod(tmp_out, mode)
os.replace(tmp_out, out_path)
finally:
if tmp_out.exists():
tmp_out.unlink()
skills/pptx/scripts/office/helpers/merge_runs.py
-199
View File
@@ -1,199 +0,0 @@
"""Merge adjacent runs with identical formatting in DOCX.
Merges adjacent <w:r> elements that have identical <w:rPr> properties.
Works on runs in paragraphs and inside tracked changes (<w:ins>, <w:del>).
Also:
- Removes rsid attributes from runs (revision metadata that doesn't affect rendering)
- Removes proofErr elements (spell/grammar markers that block merging)
"""
from pathlib import Path
import defusedxml.minidom
def merge_runs(input_dir: str) -> tuple[int, str]:
doc_xml = Path(input_dir) / "word" / "document.xml"
if not doc_xml.exists():
return 0, f"Error: {doc_xml} not found"
try:
dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8"))
root = dom.documentElement
_remove_elements(root, "proofErr")
_strip_run_rsid_attrs(root)
containers = {run.parentNode for run in _find_elements(root, "r")}
merge_count = 0
for container in containers:
merge_count += _merge_runs_in(container)
doc_xml.write_bytes(dom.toxml(encoding="UTF-8"))
return merge_count, f"Merged {merge_count} runs"
except Exception as e:
return 0, f"Error: {e}"
def _find_elements(root, tag: str) -> list:
results = []
def traverse(node):
if node.nodeType == node.ELEMENT_NODE:
name = node.localName or node.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(node)
for child in node.childNodes:
traverse(child)
traverse(root)
return results
def _get_child(parent, tag: str):
for child in parent.childNodes:
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name == tag or name.endswith(f":{tag}"):
return child
return None
def _get_children(parent, tag: str) -> list:
results = []
for child in parent.childNodes:
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(child)
return results
def _is_adjacent(elem1, elem2) -> bool:
node = elem1.nextSibling
while node:
if node == elem2:
return True
if node.nodeType == node.ELEMENT_NODE:
return False
if node.nodeType == node.TEXT_NODE and node.data.strip():
return False
node = node.nextSibling
return False
def _remove_elements(root, tag: str):
for elem in _find_elements(root, tag):
if elem.parentNode:
elem.parentNode.removeChild(elem)
def _strip_run_rsid_attrs(root):
for run in _find_elements(root, "r"):
for attr in list(run.attributes.values()):
if "rsid" in attr.name.lower():
run.removeAttribute(attr.name)
def _merge_runs_in(container) -> int:
merge_count = 0
run = _first_child_run(container)
while run:
while True:
next_elem = _next_element_sibling(run)
if next_elem and _is_run(next_elem) and _can_merge(run, next_elem):
_merge_run_content(run, next_elem)
container.removeChild(next_elem)
merge_count += 1
else:
break
_consolidate_text(run)
run = _next_sibling_run(run)
return merge_count
def _first_child_run(container):
for child in container.childNodes:
if child.nodeType == child.ELEMENT_NODE and _is_run(child):
return child
return None
def _next_element_sibling(node):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
return sibling
sibling = sibling.nextSibling
return None
def _next_sibling_run(node):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
if _is_run(sibling):
return sibling
sibling = sibling.nextSibling
return None
def _is_run(node) -> bool:
name = node.localName or node.tagName
return name == "r" or name.endswith(":r")
def _can_merge(run1, run2) -> bool:
rpr1 = _get_child(run1, "rPr")
rpr2 = _get_child(run2, "rPr")
if (rpr1 is None) != (rpr2 is None):
return False
if rpr1 is None:
return True
return rpr1.toxml() == rpr2.toxml()
def _merge_run_content(target, source):
for child in list(source.childNodes):
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name != "rPr" and not name.endswith(":rPr"):
target.appendChild(child)
def _consolidate_text(run):
t_elements = _get_children(run, "t")
for i in range(len(t_elements) - 1, 0, -1):
curr, prev = t_elements[i], t_elements[i - 1]
if _is_adjacent(prev, curr):
prev_text = prev.firstChild.data if prev.firstChild else ""
curr_text = curr.firstChild.data if curr.firstChild else ""
merged = prev_text + curr_text
if prev.firstChild:
prev.firstChild.data = merged
else:
prev.appendChild(run.ownerDocument.createTextNode(merged))
if merged.startswith(" ") or merged.endswith(" "):
prev.setAttribute("xml:space", "preserve")
elif prev.hasAttribute("xml:space"):
prev.removeAttribute("xml:space")
run.removeChild(curr)
skills/pptx/scripts/office/helpers/pptx_chart.py
+170
View File
@@ -0,0 +1,170 @@
"""Find chart XML that PowerPoint refuses but the schema accepts.
Detection only: for either fault more than one repair is valid, and only the
author knows which was meant.
"""
from __future__ import annotations
import re
from typing import Mapping
from . import part_text
_CHART_PART_RE = re.compile(r"ppt/charts/chart\d+\.xml")
_GROUPING_RE = re.compile(r"""<c:grouping\b[^>]*?\bval=["'](\w+)["']""")
_DLBL_POS_RE = re.compile(r"""<c:dLblPos\b[^>]*?\bval=["'](\w+)["']""")
def _strip_ext_lst(text: str) -> str:
out, cursor = [], 0
for lo, hi in _ext_lst_spans(text):
out.append(text[cursor:lo])
cursor = hi
out.append(text[cursor:])
return "".join(out)
_BAR_GROUP_RE = re.compile(r"<c:(bar3DChart|barChart)\b[^>]*(?<!/)>.*?</c:\1\s*>", re.DOTALL)
STACKED_GROUPINGS = frozenset({"stacked", "percentStacked"})
ILLEGAL_ON_STACKED = frozenset({"outEnd"})
LEGAL_ON_STACKED = ("ctr", "inEnd", "inBase")
def _check_stacked_label_positions(part: str, xml: str) -> list[str]:
problems: list[str] = []
for match in _BAR_GROUP_RE.finditer(xml):
block = _strip_ext_lst(match.group(0))
group = match.group(1)
grouping = _GROUPING_RE.search(block)
if grouping is None or grouping.group(1) not in STACKED_GROUPINGS:
continue
bad = [p for p in _DLBL_POS_RE.findall(block) if p in ILLEGAL_ON_STACKED]
for pos in sorted(set(bad)):
problems.append(
f'{part}: {bad.count(pos)} data label(s) use dLblPos="{pos}" on a '
f"{grouping.group(1)} {group}; PowerPoint allows only "
f"{', '.join(LEGAL_ON_STACKED)} there"
)
return problems
_ANY_CHART_GROUP_RE = re.compile(r"<c:(\w+Chart)\b[^>]*(?<!/)>.*?</c:\1\s*>", re.DOTALL)
_AXID_RE = re.compile(
r"""\s*<c:axId\b[^>]*?\bval=["'](-?\d+)["']\s*(?:/>|>\s*</c:axId\s*>)"""
)
_AXIS_DECL_RE = re.compile(
r"""<c:(catAx|valAx|serAx|dateAx)\b[^>]*(?<!/)>\s*<c:axId\b[^>]*?\bval=["'](-?\d+)["']"""
)
AXID_LIMIT = {
"barChart": 2, "lineChart": 2, "areaChart": 2, "scatterChart": 2,
"bubbleChart": 2, "radarChart": 2, "stockChart": 2,
"bar3DChart": 3, "line3DChart": 3, "area3DChart": 3,
"surfaceChart": 3, "surface3DChart": 3,
}
AXID_MINIMUM = {
"barChart": 2, "lineChart": 2, "areaChart": 2, "scatterChart": 2,
"bubbleChart": 2, "radarChart": 2, "stockChart": 2,
"bar3DChart": 2, "area3DChart": 2, "surfaceChart": 2,
"line3DChart": 3, "surface3DChart": 3,
}
def _declared_axes(xml: str) -> dict[str, list[str]]:
axes: dict[str, list[str]] = {}
for kind, axid in _AXIS_DECL_RE.findall(xml):
axes.setdefault(kind, []).append(axid)
return axes
def _canonical_ids(axes: dict[str, list[str]], limit: int) -> list[str] | None:
category = axes.get("catAx", []) + axes.get("dateAx", [])
value = axes.get("valAx", [])
series = axes.get("serAx", [])
if len(category) != 1 or len(value) != 1 or len(series) > 1:
return None
ids = [category[0], value[0]]
if limit >= 3 and series:
ids.append(series[0])
return ids
def _undeclared_axes(kind: str, block: str, axes: dict[str, list[str]]) -> list[str] | None:
if kind not in AXID_LIMIT:
return None
ids = _AXID_RE.findall(block)
declared = {i for group in axes.values() for i in group}
if len([i for i in ids if i in declared]) >= 2:
return None
return ids
def _check_chart_axis_references(part: str, xml: str) -> list[str]:
axes = _declared_axes(xml)
problems: list[str] = []
declared = {i for group in axes.values() for i in group}
for match in _ANY_CHART_GROUP_RE.finditer(xml):
kind, block = match.group(1), match.group(0)
ids = _undeclared_axes(kind, block, axes)
if ids is None:
continue
if not ids:
problems.append(
f"{part}: <c:{kind}> declares no <c:axId> this part can resolve; a chart "
f"group needs {AXID_MINIMUM[kind]}, and PowerPoint discards one with fewer"
)
continue
dead = [i for i in ids if i not in declared]
canonical = _canonical_ids(axes, AXID_LIMIT[kind])
if canonical is not None and len(canonical) >= AXID_MINIMUM[kind]:
hint = f"Fix: point them at the axes this part declares ({', '.join(canonical)})"
else:
hint = ("Fix: the part declares several axes of a kind -- declare the "
"secondary axes the series expects, or drop them")
detail = (f"of which {', '.join(dead)} name no declared axis"
if dead else f"only {len(ids)} of which this part declares")
problems.append(
f"{part}: <c:{kind}> references axId {', '.join(ids)}, {detail}, "
f"leaving fewer than two live axes; PowerPoint discards the chart. {hint}"
)
return problems
def _ext_lst_spans(text: str) -> list[tuple[int, int]]:
spans: list[tuple[int, int]] = []
depth = 0
start = 0
for match in re.finditer(r"<(/?)c:extLst\b[^>]*?(/?)>", text):
closing, self_closing = match.group(1), match.group(2)
if self_closing:
continue
if closing:
depth -= 1
if depth == 0:
spans.append((start, match.end()))
else:
if depth == 0:
start = match.start()
depth += 1
return spans
CHART_CHECKS = (_check_stacked_label_positions, _check_chart_axis_references)
def find_chart_problems(files: Mapping[str, bytes]) -> list[str]:
problems: list[str] = []
for part in sorted(n for n in files if _CHART_PART_RE.fullmatch(n)):
xml = part_text(files[part])
for check in CHART_CHECKS:
problems.extend(check(part, xml))
return problems
skills/pptx/scripts/office/helpers/pptx_slide.py
+60
View File
@@ -0,0 +1,60 @@
"""Pick the slide-XML schema errors PowerPoint refuses the file over.
A denylist over lxml's messages, so an unrecognised error class is a miss rather
than a false alarm.
"""
from __future__ import annotations
import re
SLIDE_PART_RE = re.compile(
r"ppt/(slides|slideLayouts|slideMasters|notesSlides|notesMasters|handoutMasters)"
r"/[^/]+\.xml"
)
FATAL_SLIDE_ERRORS: tuple[tuple[re.Pattern[str], str], ...] = (
(
re.compile(r"\}tableStyleId': This element is not expected"),
"two <a:tableStyleId> in one <a:tblPr> (the schema allows one)",
),
(
re.compile(r"\}srgbClr', attribute 'val'"),
"a colour that is not six hex digits",
),
(
re.compile(r"\}txBody': Missing child element"),
"a <p:txBody> with no children",
),
(
re.compile(r"\}miter', attribute 'lim'"),
'a line join with lim="NaN"',
),
(
re.compile(r"\}uLnTx': This element is not expected"),
"<a:uLnTx> in a position the schema forbids",
),
(
re.compile(r"\}overrideClrMapping': This element is not expected"),
"<p:overrideClrMapping> in a position the schema forbids",
),
(
re.compile(r"\}nvGrpSpPr': Missing child element"),
"a <p:nvGrpSpPr> with no children",
),
)
def is_schema_verdict(error: str) -> bool:
return error.startswith("Element ")
def fatal_slide_errors(errors: set[str]) -> list[str]:
out = []
for error in sorted(errors):
for pattern, meaning in FATAL_SLIDE_ERRORS:
if pattern.search(error):
out.append(f"{meaning}: {error}")
break
return out
skills/pptx/scripts/office/helpers/pptx_theme.py
+114
View File
@@ -0,0 +1,114 @@
"""Find masters sharing a theme part in the way PowerPoint refuses to open.
Reports only; the fix is to move <p:notesMasterIdLst> back to directly after
<p:sldIdLst> in ppt/presentation.xml.
"""
from __future__ import annotations
import posixpath
import re
from typing import Mapping
from . import part_text
THEME_REL_TYPE = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/theme"
_MASTER_RE = re.compile(
r"^ppt/(?P<group>slideMasters|notesMasters|handoutMasters)/"
r"(?:slide|notes|handout)Master(?P<num>\d+)\.xml$"
)
_GROUP_ORDER = {"slideMasters": 0, "notesMasters": 1, "handoutMasters": 2}
_RELATIONSHIP_RE = re.compile(
r"<Relationship\b[^>]*?(?:/>|>.*?</Relationship\s*>)", re.DOTALL
)
def _sort_key(name: str) -> tuple[int, int]:
m = _MASTER_RE.match(name)
assert m is not None
return (_GROUP_ORDER[m.group("group")], int(m.group("num")))
def _rels_path(part: str) -> str:
directory, base = posixpath.split(part)
return f"{directory}/_rels/{base}.rels"
def _resolve(rels_path: str, target: str) -> str:
if target.startswith("/"):
return target.lstrip("/")
part_dir = posixpath.dirname(posixpath.dirname(rels_path))
return posixpath.normpath(posixpath.join(part_dir, target))
def _theme_rel(files: Mapping[str, bytes], master: str):
rels_path = _rels_path(master)
rels = files.get(rels_path)
if rels is None:
return None
for element in _RELATIONSHIP_RE.findall(part_text(rels)):
if f'Type="{THEME_REL_TYPE}"' not in element:
continue
target = re.search(r'\bTarget="([^"]+)"', element)
if target is None:
continue
return rels_path, element, _resolve(rels_path, target.group(1))
return None
def _masters(files: Mapping[str, bytes]) -> list[str]:
return sorted((n for n in files if _MASTER_RE.match(n)), key=_sort_key)
_PRESENTATION = "ppt/presentation.xml"
_NOTES_MASTERS = "ppt/notesMasters/"
_IGNORABLE_RE = re.compile(r"<!--.*?-->|<\?.*?\?>", re.DOTALL)
_AFTER_SLDIDLST_RE = re.compile(
r"<p:sldIdLst\b(?:[^>]*/>|[^>]*>.*?</p:sldIdLst\s*>)\s*(<[^>\s/]+)", re.DOTALL
)
def _notes_master_share_is_inert(files: Mapping[str, bytes]) -> bool:
data = files.get(_PRESENTATION)
if data is None:
return False
match = _AFTER_SLDIDLST_RE.search(_IGNORABLE_RE.sub("", part_text(data)))
return match is not None and match.group(1) == "<p:notesMasterIdLst"
def _shares(files: Mapping[str, bytes]):
owner: dict[str, str] = {}
for master in _masters(files):
found = _theme_rel(files, master)
if found is None:
continue
rels_path, element, theme = found
if theme not in files:
continue
if theme in owner:
yield master, rels_path, element, theme, owner[theme]
else:
owner[theme] = master
def _is_inert(master: str, inert_notes: bool) -> bool:
return inert_notes and master.startswith(_NOTES_MASTERS)
def find_shared_master_themes(files: Mapping[str, bytes]) -> list[str]:
return [
f"{master} shares {theme} with {first}"
for master, _, _, theme, first in _shares(files)
]
def live_shared_master_themes(files: Mapping[str, bytes]) -> list[str]:
inert_notes = _notes_master_share_is_inert(files)
return [
f"{master} shares {theme} with {first}"
for master, _, _, theme, first in _shares(files)
if not _is_inert(master, inert_notes)
]
skills/pptx/scripts/office/helpers/simplify_redlines.py
-197
View File
@@ -1,197 +0,0 @@
"""Simplify tracked changes by merging adjacent w:ins or w:del elements.
Merges adjacent <w:ins> elements from the same author into a single element.
Same for <w:del> elements. This makes heavily-redlined documents easier to
work with by reducing the number of tracked change wrappers.
Rules:
- Only merges w:ins with w:ins, w:del with w:del (same element type)
- Only merges if same author (ignores timestamp differences)
- Only merges if truly adjacent (only whitespace between them)
"""
import xml.etree.ElementTree as ET
import zipfile
from pathlib import Path
import defusedxml.minidom
WORD_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
def simplify_redlines(input_dir: str) -> tuple[int, str]:
doc_xml = Path(input_dir) / "word" / "document.xml"
if not doc_xml.exists():
return 0, f"Error: {doc_xml} not found"
try:
dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8"))
root = dom.documentElement
merge_count = 0
containers = _find_elements(root, "p") + _find_elements(root, "tc")
for container in containers:
merge_count += _merge_tracked_changes_in(container, "ins")
merge_count += _merge_tracked_changes_in(container, "del")
doc_xml.write_bytes(dom.toxml(encoding="UTF-8"))
return merge_count, f"Simplified {merge_count} tracked changes"
except Exception as e:
return 0, f"Error: {e}"
def _merge_tracked_changes_in(container, tag: str) -> int:
merge_count = 0
tracked = [
child
for child in container.childNodes
if child.nodeType == child.ELEMENT_NODE and _is_element(child, tag)
]
if len(tracked) < 2:
return 0
i = 0
while i < len(tracked) - 1:
curr = tracked[i]
next_elem = tracked[i + 1]
if _can_merge_tracked(curr, next_elem):
_merge_tracked_content(curr, next_elem)
container.removeChild(next_elem)
tracked.pop(i + 1)
merge_count += 1
else:
i += 1
return merge_count
def _is_element(node, tag: str) -> bool:
name = node.localName or node.tagName
return name == tag or name.endswith(f":{tag}")
def _get_author(elem) -> str:
author = elem.getAttribute("w:author")
if not author:
for attr in elem.attributes.values():
if attr.localName == "author" or attr.name.endswith(":author"):
return attr.value
return author
def _can_merge_tracked(elem1, elem2) -> bool:
if _get_author(elem1) != _get_author(elem2):
return False
node = elem1.nextSibling
while node and node != elem2:
if node.nodeType == node.ELEMENT_NODE:
return False
if node.nodeType == node.TEXT_NODE and node.data.strip():
return False
node = node.nextSibling
return True
def _merge_tracked_content(target, source):
while source.firstChild:
child = source.firstChild
source.removeChild(child)
target.appendChild(child)
def _find_elements(root, tag: str) -> list:
results = []
def traverse(node):
if node.nodeType == node.ELEMENT_NODE:
name = node.localName or node.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(node)
for child in node.childNodes:
traverse(child)
traverse(root)
return results
def get_tracked_change_authors(doc_xml_path: Path) -> dict[str, int]:
if not doc_xml_path.exists():
return {}
try:
tree = ET.parse(doc_xml_path)
root = tree.getroot()
except ET.ParseError:
return {}
namespaces = {"w": WORD_NS}
author_attr = f"{{{WORD_NS}}}author"
authors: dict[str, int] = {}
for tag in ["ins", "del"]:
for elem in root.findall(f".//w:{tag}", namespaces):
author = elem.get(author_attr)
if author:
authors[author] = authors.get(author, 0) + 1
return authors
def _get_authors_from_docx(docx_path: Path) -> dict[str, int]:
try:
with zipfile.ZipFile(docx_path, "r") as zf:
if "word/document.xml" not in zf.namelist():
return {}
with zf.open("word/document.xml") as f:
tree = ET.parse(f)
root = tree.getroot()
namespaces = {"w": WORD_NS}
author_attr = f"{{{WORD_NS}}}author"
authors: dict[str, int] = {}
for tag in ["ins", "del"]:
for elem in root.findall(f".//w:{tag}", namespaces):
author = elem.get(author_attr)
if author:
authors[author] = authors.get(author, 0) + 1
return authors
except (zipfile.BadZipFile, ET.ParseError):
return {}
def infer_author(modified_dir: Path, original_docx: Path, default: str = "Claude") -> str:
modified_xml = modified_dir / "word" / "document.xml"
modified_authors = get_tracked_change_authors(modified_xml)
if not modified_authors:
return default
original_authors = _get_authors_from_docx(original_docx)
new_changes: dict[str, int] = {}
for author, count in modified_authors.items():
original_count = original_authors.get(author, 0)
diff = count - original_count
if diff > 0:
new_changes[author] = diff
if not new_changes:
return default
if len(new_changes) == 1:
return next(iter(new_changes))
raise ValueError(
f"Multiple authors added new changes: {new_changes}. "
"Cannot infer which author to validate."
)
skills/pptx/scripts/office/pack.py
-159
View File
@@ -1,159 +0,0 @@
"""Pack a directory into a DOCX, PPTX, or XLSX file.
Validates with auto-repair, condenses XML formatting, and creates the Office file.
Usage:
python pack.py <input_directory> <output_file> [--original <file>] [--validate true|false]
Examples:
python pack.py unpacked/ output.docx --original input.docx
python pack.py unpacked/ output.pptx --validate false
"""
import argparse
import sys
import shutil
import tempfile
import zipfile
from pathlib import Path
import defusedxml.minidom
from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator
def pack(
input_directory: str,
output_file: str,
original_file: str | None = None,
validate: bool = True,
infer_author_func=None,
) -> tuple[None, str]:
input_dir = Path(input_directory)
output_path = Path(output_file)
suffix = output_path.suffix.lower()
if not input_dir.is_dir():
return None, f"Error: {input_dir} is not a directory"
if suffix not in {".docx", ".pptx", ".xlsx"}:
return None, f"Error: {output_file} must be a .docx, .pptx, or .xlsx file"
if validate and original_file:
original_path = Path(original_file)
if original_path.exists():
success, output = _run_validation(
input_dir, original_path, suffix, infer_author_func
)
if output:
print(output)
if not success:
return None, f"Error: Validation failed for {input_dir}"
with tempfile.TemporaryDirectory() as temp_dir:
temp_content_dir = Path(temp_dir) / "content"
shutil.copytree(input_dir, temp_content_dir)
for pattern in ["*.xml", "*.rels"]:
for xml_file in temp_content_dir.rglob(pattern):
_condense_xml(xml_file)
output_path.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zf:
for f in temp_content_dir.rglob("*"):
if f.is_file():
zf.write(f, f.relative_to(temp_content_dir))
return None, f"Successfully packed {input_dir} to {output_file}"
def _run_validation(
unpacked_dir: Path,
original_file: Path,
suffix: str,
infer_author_func=None,
) -> tuple[bool, str | None]:
output_lines = []
validators = []
if suffix == ".docx":
author = "Claude"
if infer_author_func:
try:
author = infer_author_func(unpacked_dir, original_file)
except ValueError as e:
print(f"Warning: {e} Using default author 'Claude'.", file=sys.stderr)
validators = [
DOCXSchemaValidator(unpacked_dir, original_file),
RedliningValidator(unpacked_dir, original_file, author=author),
]
elif suffix == ".pptx":
validators = [PPTXSchemaValidator(unpacked_dir, original_file)]
if not validators:
return True, None
total_repairs = sum(v.repair() for v in validators)
if total_repairs:
output_lines.append(f"Auto-repaired {total_repairs} issue(s)")
success = all(v.validate() for v in validators)
if success:
output_lines.append("All validations PASSED!")
return success, "\n".join(output_lines) if output_lines else None
def _condense_xml(xml_file: Path) -> None:
try:
with open(xml_file, encoding="utf-8") as f:
dom = defusedxml.minidom.parse(f)
for element in dom.getElementsByTagName("*"):
if element.tagName.endswith(":t"):
continue
for child in list(element.childNodes):
if (
child.nodeType == child.TEXT_NODE
and child.nodeValue
and child.nodeValue.strip() == ""
) or child.nodeType == child.COMMENT_NODE:
element.removeChild(child)
xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
except Exception as e:
print(f"ERROR: Failed to parse {xml_file.name}: {e}", file=sys.stderr)
raise
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Pack a directory into a DOCX, PPTX, or XLSX file"
)
parser.add_argument("input_directory", help="Unpacked Office document directory")
parser.add_argument("output_file", help="Output Office file (.docx/.pptx/.xlsx)")
parser.add_argument(
"--original",
help="Original file for validation comparison",
)
parser.add_argument(
"--validate",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Run validation with auto-repair (default: true)",
)
args = parser.parse_args()
_, message = pack(
args.input_directory,
args.output_file,
original_file=args.original,
validate=args.validate,
)
print(message)
if "Error" in message:
sys.exit(1)
skills/pptx/scripts/office/soffice.py
+17 -8
View File
@@ -4,20 +4,23 @@ sockets may be blocked (e.g., sandboxed VMs). Detects the restriction
at runtime and applies an LD_PRELOAD shim if needed. at runtime and applies an LD_PRELOAD shim if needed.
Usage: Usage:
from office.soffice import run_soffice, get_soffice_env from office.soffice import run_soffice
# Option 1 – run soffice directly
result = run_soffice(["--headless", "--convert-to", "pdf", "input.docx"]) result = run_soffice(["--headless", "--convert-to", "pdf", "input.docx"])
# Option 2 – get env dict for your own subprocess calls Call soffice through run_soffice, not through subprocess with get_soffice_env():
env = get_soffice_env() the env dict carries the shim but names no user profile, and a non-root sandbox
subprocess.run(["soffice", ...], env=env) cannot bootstrap the default one -- soffice aborts with "User installation could
not be completed" and converts nothing. get_soffice_env() stays public for the
callers that build their own argv (they must pass -env:UserInstallation too).
""" """
import contextlib
import os import os
import socket import socket
import subprocess import subprocess
import tempfile import tempfile
from collections.abc import Iterable
from pathlib import Path from pathlib import Path
@@ -32,9 +35,15 @@ def get_soffice_env() -> dict:
return env return env
def run_soffice(args: list[str], **kwargs) -> subprocess.CompletedProcess: def run_soffice(args: Iterable[str], **kwargs) -> subprocess.CompletedProcess:
env = get_soffice_env() args = list(args)
return subprocess.run(["soffice"] + args, env=env, **kwargs) with contextlib.ExitStack() as stack:
if not any(str(a).startswith("-env:UserInstallation") for a in args):
profile = stack.enter_context(
tempfile.TemporaryDirectory(prefix="lo_profile_", ignore_cleanup_errors=True)
)
args = [f"-env:UserInstallation={Path(profile).as_uri()}"] + args
return subprocess.run(["soffice"] + args, env=get_soffice_env(), **kwargs)
skills/pptx/scripts/office/unpack.py
-132
View File
@@ -1,132 +0,0 @@
"""Unpack Office files (DOCX, PPTX, XLSX) for editing.
Extracts the ZIP archive, pretty-prints XML files, and optionally:
- Merges adjacent runs with identical formatting (DOCX only)
- Simplifies adjacent tracked changes from same author (DOCX only)
Usage:
python unpack.py <office_file> <output_dir> [options]
Examples:
python unpack.py document.docx unpacked/
python unpack.py presentation.pptx unpacked/
python unpack.py document.docx unpacked/ --merge-runs false
"""
import argparse
import sys
import zipfile
from pathlib import Path
import defusedxml.minidom
from helpers.merge_runs import merge_runs as do_merge_runs
from helpers.simplify_redlines import simplify_redlines as do_simplify_redlines
SMART_QUOTE_REPLACEMENTS = {
"\u201c": "&#x201C;",
"\u201d": "&#x201D;",
"\u2018": "&#x2018;",
"\u2019": "&#x2019;",
}
def unpack(
input_file: str,
output_directory: str,
merge_runs: bool = True,
simplify_redlines: bool = True,
) -> tuple[None, str]:
input_path = Path(input_file)
output_path = Path(output_directory)
suffix = input_path.suffix.lower()
if not input_path.exists():
return None, f"Error: {input_file} does not exist"
if suffix not in {".docx", ".pptx", ".xlsx"}:
return None, f"Error: {input_file} must be a .docx, .pptx, or .xlsx file"
try:
output_path.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(input_path, "r") as zf:
zf.extractall(output_path)
xml_files = list(output_path.rglob("*.xml")) + list(output_path.rglob("*.rels"))
for xml_file in xml_files:
_pretty_print_xml(xml_file)
message = f"Unpacked {input_file} ({len(xml_files)} XML files)"
if suffix == ".docx":
if simplify_redlines:
simplify_count, _ = do_simplify_redlines(str(output_path))
message += f", simplified {simplify_count} tracked changes"
if merge_runs:
merge_count, _ = do_merge_runs(str(output_path))
message += f", merged {merge_count} runs"
for xml_file in xml_files:
_escape_smart_quotes(xml_file)
return None, message
except zipfile.BadZipFile:
return None, f"Error: {input_file} is not a valid Office file"
except Exception as e:
return None, f"Error unpacking: {e}"
def _pretty_print_xml(xml_file: Path) -> None:
try:
content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content)
xml_file.write_bytes(dom.toprettyxml(indent=" ", encoding="utf-8"))
except Exception:
pass
def _escape_smart_quotes(xml_file: Path) -> None:
try:
content = xml_file.read_text(encoding="utf-8")
for char, entity in SMART_QUOTE_REPLACEMENTS.items():
content = content.replace(char, entity)
xml_file.write_text(content, encoding="utf-8")
except Exception:
pass
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Unpack an Office file (DOCX, PPTX, XLSX) for editing"
)
parser.add_argument("input_file", help="Office file to unpack")
parser.add_argument("output_directory", help="Output directory")
parser.add_argument(
"--merge-runs",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Merge adjacent runs with identical formatting (DOCX only, default: true)",
)
parser.add_argument(
"--simplify-redlines",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Merge adjacent tracked changes from same author (DOCX only, default: true)",
)
args = parser.parse_args()
_, message = unpack(
args.input_file,
args.output_directory,
merge_runs=args.merge_runs,
simplify_redlines=args.simplify_redlines,
)
print(message)
if "Error" in message:
sys.exit(1)
skills/pptx/scripts/office/validate.py
+89 -27
View File
@@ -6,7 +6,7 @@ Usage:
The first argument can be either: The first argument can be either:
- An unpacked directory containing the Office document XML files - An unpacked directory containing the Office document XML files
- A packed Office file (.docx/.pptx/.xlsx) which will be unpacked to a temp directory - A packed Office file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx template) which will be unpacked to a temp directory
Auto-repair fixes: Auto-repair fixes:
- paraId/durableId values that exceed OOXML limits - paraId/durableId values that exceed OOXML limits
@@ -19,20 +19,43 @@ import tempfile
import zipfile import zipfile
from pathlib import Path from pathlib import Path
import defusedxml.ElementTree as ET
from defusedxml.common import DefusedXmlException
from helpers import OOXML_FAMILY, rezip, safe_extract
from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator
WORD_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
def _fail(message: str):
print(f"Error: {message}", file=sys.stderr)
sys.exit(2)
def _has_tracked_changes(unpacked_dir: Path) -> bool:
document = unpacked_dir / "word" / "document.xml"
if not document.is_file():
return False
try:
root = ET.parse(document).getroot()
except (ET.ParseError, DefusedXmlException):
return False
tracked = {f"{{{WORD_NS}}}ins", f"{{{WORD_NS}}}del"}
return any(elem.tag in tracked for elem in root.iter())
def main(): def main():
parser = argparse.ArgumentParser(description="Validate Office document XML files") parser = argparse.ArgumentParser(description="Validate Office document XML files")
parser.add_argument( parser.add_argument(
"path", "path",
help="Path to unpacked directory or packed Office file (.docx/.pptx/.xlsx)", help="Path to unpacked directory or packed Office file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx)",
) )
parser.add_argument( parser.add_argument(
"--original", "--original",
required=False, required=False,
default=None, default=None,
help="Path to original file (.docx/.pptx/.xlsx). If omitted, all XSD errors are reported and redlining validation is skipped.", help="Path to original file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx). If omitted, all XSD errors are reported and redlining validation is skipped.",
) )
parser.add_argument( parser.add_argument(
"-v", "-v",
@@ -43,63 +66,102 @@ def main():
parser.add_argument( parser.add_argument(
"--auto-repair", "--auto-repair",
action="store_true", action="store_true",
help="Automatically repair common issues (hex IDs, whitespace preservation)", help="Automatically repair common issues (hex IDs, whitespace preservation). "
"Modifies the input in place: repairs to a packed file are written back to it.",
) )
parser.add_argument( parser.add_argument(
"--author", "--author",
default="Claude", default=None,
help="Author name for redlining validation (default: Claude)", help="The name you are redlining under. Passing it turns on the "
"tracked-change check: any text differing from --original without a "
"<w:ins>/<w:del> recording it is reported. Untracked edits carry no "
"author, so the check covers them whoever made them — the name marks "
"the run as redlining work and is not used to filter. Requires "
"--original; docx only.",
) )
args = parser.parse_args() args = parser.parse_args()
if args.author is not None and not args.original:
_fail("--author requires --original")
path = Path(args.path) path = Path(args.path)
assert path.exists(), f"Error: {path} does not exist" if not path.exists():
_fail(f"{path} does not exist")
original_file = None original_file = None
if args.original: if args.original:
original_file = Path(args.original) original_file = Path(args.original)
assert original_file.is_file(), f"Error: {original_file} is not a file" if not original_file.is_file():
assert original_file.suffix.lower() in [".docx", ".pptx", ".xlsx"], ( _fail(f"{original_file} is not a file")
f"Error: {original_file} must be a .docx, .pptx, or .xlsx file" if original_file.suffix.lower() not in OOXML_FAMILY:
_fail(f"{original_file} must be one of: {', '.join(sorted(OOXML_FAMILY))}")
family = OOXML_FAMILY.get((original_file or path).suffix.lower())
if family is None:
_fail(
f"Cannot determine file type from {path}. Use --original or provide one of: {', '.join(sorted(OOXML_FAMILY))}."
) )
file_extension = (original_file or path).suffix.lower() if args.author is not None and family != "docx":
assert file_extension in [".docx", ".pptx", ".xlsx"], ( _fail(f"--author only applies to docx files, not {family}")
f"Error: Cannot determine file type from {path}. Use --original or provide a .docx/.pptx/.xlsx file."
)
if path.is_file() and path.suffix.lower() in [".docx", ".pptx", ".xlsx"]: packed_file = None
temp_dir = tempfile.mkdtemp() temp_dir_ctx = None
with zipfile.ZipFile(path, "r") as zf: if path.is_file() and path.suffix.lower() in OOXML_FAMILY:
zf.extractall(temp_dir) packed_file = path
unpacked_dir = Path(temp_dir) temp_dir_ctx = tempfile.TemporaryDirectory()
unpacked_dir = Path(temp_dir_ctx.name)
try:
with zipfile.ZipFile(path, "r") as zf:
safe_extract(zf, unpacked_dir)
except (zipfile.BadZipFile, ValueError, OSError) as e:
_fail(f"cannot unpack {path}: {e}")
else: else:
assert path.is_dir(), f"Error: {path} is not a directory or Office file" if not path.is_dir():
_fail(f"{path} is not a directory or Office file")
unpacked_dir = path unpacked_dir = path
match file_extension: match family:
case ".docx": case "docx":
validators = [ validators = [
DOCXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), DOCXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose),
] ]
if original_file: if args.author is not None:
validators.append( validators.append(
RedliningValidator(unpacked_dir, original_file, verbose=args.verbose, author=args.author) RedliningValidator(unpacked_dir, original_file, verbose=args.verbose)
) )
case ".pptx": elif original_file and _has_tracked_changes(unpacked_dir):
print(
"Note: this document has tracked changes; they were not "
"checked against the original (pass --author to check)."
)
case "pptx":
validators = [ validators = [
PPTXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), PPTXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose),
] ]
case "xlsx":
exts = ", ".join(k for k, v in sorted(OOXML_FAMILY.items()) if v == "xlsx")
print(
f"No XSD schema validation is performed for xlsx-family files ({exts}). "
"For formula-error checking, use scripts/recalc.py instead."
)
sys.exit(0)
case _: case _:
print(f"Error: Validation not supported for file type {file_extension}") print(f"Error: Validation not supported for file type {family}")
sys.exit(1) sys.exit(1)
if args.auto_repair: if args.auto_repair:
total_repairs = sum(v.repair() for v in validators) total_repairs = sum(v.repair() for v in validators)
if total_repairs: if total_repairs:
print(f"Auto-repaired {total_repairs} issue(s)") print(f"Auto-repaired {total_repairs} issue(s)")
if packed_file is not None:
rezip(unpacked_dir, packed_file)
print(f"Wrote repaired file to {packed_file}")
success = all(v.validate() for v in validators) success = all([v.validate() for v in validators])
if temp_dir_ctx is not None:
temp_dir_ctx.cleanup()
if success: if success:
print("All validations PASSED!") print("All validations PASSED!")
skills/pptx/scripts/office/validators/base.py
+48 -20
View File
@@ -6,8 +6,20 @@ import re
from pathlib import Path from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
from functools import lru_cache
import lxml.etree import lxml.etree
from helpers import safe_extract
@lru_cache(maxsize=None)
def _load_schema(schema_path: str):
with open(schema_path, "rb") as xsd_file:
xsd_doc = lxml.etree.parse(
xsd_file, parser=lxml.etree.XMLParser(), base_url=schema_path
)
return lxml.etree.XMLSchema(xsd_doc)
class BaseSchemaValidator: class BaseSchemaValidator:
@@ -119,21 +131,28 @@ class BaseSchemaValidator:
try: try:
content = xml_file.read_text(encoding="utf-8") content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content) dom = defusedxml.minidom.parseString(content)
modified = False pending = []
for elem in dom.getElementsByTagName("*"): for elem in dom.getElementsByTagName("*"):
if elem.tagName.endswith(":t") and elem.firstChild: local_name = elem.tagName.rsplit(":", 1)[-1]
text = elem.firstChild.nodeValue if local_name in ("t", "delText", "instrText", "delInstrText"):
if text and (text.startswith((' ', '\t')) or text.endswith((' ', '\t'))): text = "".join(
child.data
for child in elem.childNodes
if child.nodeType in (child.TEXT_NODE, child.CDATA_SECTION_NODE)
)
ws = (" ", "\t", "\n", "\r")
if text and (text.startswith(ws) or text.endswith(ws)):
if elem.getAttribute("xml:space") != "preserve": if elem.getAttribute("xml:space") != "preserve":
elem.setAttribute("xml:space", "preserve") elem.setAttribute("xml:space", "preserve")
text_preview = repr(text[:30]) + "..." if len(text) > 30 else repr(text) text_preview = repr(text[:30]) + "..." if len(text) > 30 else repr(text)
print(f" Repaired: {xml_file.name}: Added xml:space='preserve' to {elem.tagName}: {text_preview}") pending.append(f" Repaired: {xml_file.name}: Added xml:space='preserve' to {elem.tagName}: {text_preview}")
repairs += 1
modified = True
if modified: if pending:
xml_file.write_bytes(dom.toxml(encoding="UTF-8")) xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
for message in pending:
print(message)
repairs += len(pending)
except Exception: except Exception:
pass pass
@@ -212,6 +231,8 @@ class BaseSchemaValidator:
elem.getparent().remove(elem) elem.getparent().remove(elem)
for elem in root.iter(): for elem in root.iter():
if not hasattr(elem, "tag") or callable(elem.tag):
continue
tag = ( tag = (
elem.tag.split("}")[-1].lower() elem.tag.split("}")[-1].lower()
if "}" in elem.tag if "}" in elem.tag
@@ -326,6 +347,8 @@ class BaseSchemaValidator:
namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE}, namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE},
): ):
target = rel.get("Target") target = rel.get("Target")
if rel.get("TargetMode") == "External":
continue
if target and not target.startswith( if target and not target.startswith(
("http", "mailto:") ("http", "mailto:")
): ):
@@ -423,6 +446,8 @@ class BaseSchemaValidator:
r_ns = self.OFFICE_RELATIONSHIPS_NAMESPACE r_ns = self.OFFICE_RELATIONSHIPS_NAMESPACE
rid_attrs_to_check = ["id", "embed", "link"] rid_attrs_to_check = ["id", "embed", "link"]
for elem in xml_root.iter(): for elem in xml_root.iter():
if not hasattr(elem, "tag") or callable(elem.tag):
continue
for attr_name in rid_attrs_to_check: for attr_name in rid_attrs_to_check:
rid_attr = elem.get(f"{{{r_ns}}}{attr_name}") rid_attr = elem.get(f"{{{r_ns}}}{attr_name}")
if not rid_attr: if not rid_attr:
@@ -747,18 +772,16 @@ class BaseSchemaValidator:
return xml_doc return xml_doc
def _validate_single_file_xsd(self, xml_file, base_path): def _preprocess_for_schema(self, xml_doc, relative_path):
schema_path = self._get_schema_path(xml_file) return xml_doc
def _validate_single_file_xsd(self, xml_file, base_path, schema_path=None):
schema_path = schema_path or self._get_schema_path(xml_file)
if not schema_path: if not schema_path:
return None, None return None, None
try: try:
with open(schema_path, "rb") as xsd_file: schema = _load_schema(str(schema_path))
parser = lxml.etree.XMLParser()
xsd_doc = lxml.etree.parse(
xsd_file, parser=parser, base_url=str(schema_path)
)
schema = lxml.etree.XMLSchema(xsd_doc)
with open(xml_file, "r") as f: with open(xml_file, "r") as f:
xml_doc = lxml.etree.parse(f) xml_doc = lxml.etree.parse(f)
@@ -773,6 +796,8 @@ class BaseSchemaValidator:
): ):
xml_doc = self._clean_ignorable_namespaces(xml_doc) xml_doc = self._clean_ignorable_namespaces(xml_doc)
xml_doc = self._preprocess_for_schema(xml_doc, relative_path)
if schema.validate(xml_doc): if schema.validate(xml_doc):
return True, set() return True, set()
else: else:
@@ -784,7 +809,7 @@ class BaseSchemaValidator:
except Exception as e: except Exception as e:
return False, {str(e)} return False, {str(e)}
def _get_original_file_errors(self, xml_file): def _get_original_file_errors(self, xml_file, schema_path=None):
if self.original_file is None: if self.original_file is None:
return set() return set()
@@ -798,8 +823,11 @@ class BaseSchemaValidator:
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir) temp_path = Path(temp_dir)
with zipfile.ZipFile(self.original_file, "r") as zip_ref: try:
zip_ref.extractall(temp_path) with zipfile.ZipFile(self.original_file, "r") as zip_ref:
safe_extract(zip_ref, temp_path)
except (zipfile.BadZipFile, ValueError, OSError):
return set()
original_xml_file = temp_path / relative_path original_xml_file = temp_path / relative_path
@@ -807,7 +835,7 @@ class BaseSchemaValidator:
return set() return set()
is_valid, errors = self._validate_single_file_xsd( is_valid, errors = self._validate_single_file_xsd(
original_xml_file, temp_path original_xml_file, temp_path, schema_path=schema_path
) )
return errors if errors else set() return errors if errors else set()
skills/pptx/scripts/office/validators/docx.py
+56 -36
View File
@@ -6,10 +6,13 @@ import random
import re import re
import tempfile import tempfile
import zipfile import zipfile
from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
import lxml.etree import lxml.etree
from helpers import safe_extract
from .base import BaseSchemaValidator from .base import BaseSchemaValidator
@@ -186,7 +189,7 @@ class DOCXSchemaValidator(BaseSchemaValidator):
try: try:
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
with zipfile.ZipFile(original, "r") as zip_ref: with zipfile.ZipFile(original, "r") as zip_ref:
zip_ref.extractall(temp_dir) safe_extract(zip_ref, Path(temp_dir))
doc_xml_path = temp_dir + "/word/document.xml" doc_xml_path = temp_dir + "/word/document.xml"
root = lxml.etree.parse(doc_xml_path).getroot() root = lxml.etree.parse(doc_xml_path).getroot()
@@ -241,9 +244,12 @@ class DOCXSchemaValidator(BaseSchemaValidator):
return True return True
def compare_paragraph_counts(self): def compare_paragraph_counts(self):
original_count = self.count_paragraphs_in_original()
new_count = self.count_paragraphs_in_unpacked() new_count = self.count_paragraphs_in_unpacked()
if self.original_file is None:
print(f"\nParagraphs: {new_count}")
return
original_count = self.count_paragraphs_in_original()
diff = new_count - original_count diff = new_count - original_count
diff_str = f"+{diff}" if diff > 0 else str(diff) diff_str = f"+{diff}" if diff > 0 else str(diff)
print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})") print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})")
@@ -260,9 +266,15 @@ class DOCXSchemaValidator(BaseSchemaValidator):
try: try:
for elem in lxml.etree.parse(str(xml_file)).iter(): for elem in lxml.etree.parse(str(xml_file)).iter():
if val := elem.get(para_id_attr): if val := elem.get(para_id_attr):
if self._parse_id_value(val, base=16) >= 0x80000000: try:
if self._parse_id_value(val, base=16) >= 0x80000000:
errors.append(
f" {xml_file.name}:{elem.sourceline}: paraId={val} >= 0x80000000"
)
except ValueError:
errors.append( errors.append(
f" {xml_file.name}:{elem.sourceline}: paraId={val} >= 0x80000000" f" {xml_file.name}:{elem.sourceline}: "
f"paraId={val} is not valid hex"
) )
if val := elem.get(durable_id_attr): if val := elem.get(durable_id_attr):
@@ -279,13 +291,19 @@ class DOCXSchemaValidator(BaseSchemaValidator):
f"durableId={val} must be decimal in numbering.xml" f"durableId={val} must be decimal in numbering.xml"
) )
else: else:
if self._parse_id_value(val, base=16) >= 0x7FFFFFFF: try:
if self._parse_id_value(val, base=16) >= 0x7FFFFFFF:
errors.append(
f" {xml_file.name}:{elem.sourceline}: "
f"durableId={val} >= 0x7FFFFFFF"
)
except ValueError:
errors.append( errors.append(
f" {xml_file.name}:{elem.sourceline}: " f" {xml_file.name}:{elem.sourceline}: "
f"durableId={val} >= 0x7FFFFFFF" f"durableId={val} is not valid hex"
) )
except Exception: except lxml.etree.XMLSyntaxError:
pass continue
if errors: if errors:
print(f"FAILED - {len(errors)} ID constraint violations:") print(f"FAILED - {len(errors)} ID constraint violations:")
@@ -389,52 +407,54 @@ class DOCXSchemaValidator(BaseSchemaValidator):
return repairs return repairs
def repair_durableId(self) -> int: def repair_durableId(self) -> int:
DURABLE_ID_ATTRS = ("w16cid:durableId", "w16cex:durableId")
repairs = 0 repairs = 0
renames: dict = {}
for xml_file in self.xml_files: for xml_file in self.xml_files:
try: try:
content = xml_file.read_text(encoding="utf-8") content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content) dom = defusedxml.minidom.parseString(content)
is_numbering = xml_file.name == "numbering.xml"
base = 10 if is_numbering else 16
pending = []
seen_in_file = set()
modified = False modified = False
for elem in dom.getElementsByTagName("*"): for elem in dom.getElementsByTagName("*"):
if not elem.hasAttribute("w16cid:durableId"): for attr_name in DURABLE_ID_ATTRS:
continue if not elem.hasAttribute(attr_name):
continue
durable_id = elem.getAttribute("w16cid:durableId") durable_id = elem.getAttribute(attr_name)
needs_repair = False
if xml_file.name == "numbering.xml":
try: try:
needs_repair = ( key = self._parse_id_value(durable_id, base=base)
self._parse_id_value(durable_id, base=10) >= 0x7FFFFFFF needs_repair = key >= 0x7FFFFFFF
)
except ValueError:
needs_repair = True
else:
try:
needs_repair = (
self._parse_id_value(durable_id, base=16) >= 0x7FFFFFFF
)
except ValueError: except ValueError:
key = durable_id
needs_repair = True needs_repair = True
if needs_repair: if needs_repair:
value = random.randint(1, 0x7FFFFFFE) if key in seen_in_file:
if xml_file.name == "numbering.xml": value = random.randint(1, 0x7FFFFFFE)
new_id = str(value) else:
else: seen_in_file.add(key)
new_id = f"{value:08X}" if key not in renames:
renames[key] = random.randint(1, 0x7FFFFFFE)
value = renames[key]
new_id = str(value) if is_numbering else f"{value:08X}"
elem.setAttribute("w16cid:durableId", new_id) elem.setAttribute(attr_name, new_id)
print( pending.append(
f" Repaired: {xml_file.name}: durableId {durable_id} → {new_id}" f" Repaired: {xml_file.name}: durableId {durable_id} → {new_id}"
) )
repairs += 1 modified = True
modified = True
if modified: if modified:
xml_file.write_bytes(dom.toxml(encoding="UTF-8")) xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
for message in pending:
print(message)
repairs += len(pending)
except Exception: except Exception:
pass pass
skills/pptx/scripts/office/validators/pptx.py
+173 -7
View File
@@ -3,6 +3,9 @@ Validator for PowerPoint presentation XML files against XSD schemas.
""" """
import re import re
from pathlib import Path
from helpers import opc_target, rels_source_part, safe_extract
from .base import BaseSchemaValidator from .base import BaseSchemaValidator
@@ -57,8 +60,171 @@ class PPTXSchemaValidator(BaseSchemaValidator):
if not self.validate_no_duplicate_slide_layouts(): if not self.validate_no_duplicate_slide_layouts():
all_valid = False all_valid = False
if not self.validate_master_theme_uniqueness():
all_valid = False
if not self.validate_charts():
all_valid = False
if not self.validate_slides():
all_valid = False
return all_valid return all_valid
def _package_map(self) -> dict:
wanted = []
wanted += list(self.unpacked_dir.glob("[[]Content_Types[]].xml"))
wanted += list(self.unpacked_dir.glob("ppt/presentation.xml"))
wanted += list(self.unpacked_dir.glob("ppt/theme/*.xml"))
wanted += list(self.unpacked_dir.glob("ppt/theme/_rels/*.rels"))
wanted += list(self.unpacked_dir.glob("ppt/charts/chart*.xml"))
for group in ("slideMasters", "notesMasters", "handoutMasters"):
wanted += list(self.unpacked_dir.glob(f"ppt/{group}/*.xml"))
wanted += list(self.unpacked_dir.glob(f"ppt/{group}/_rels/*.rels"))
return {
p.relative_to(self.unpacked_dir).as_posix(): p.read_bytes()
for p in wanted
if p.is_file()
}
def validate_master_theme_uniqueness(self):
from helpers.pptx_theme import _NOTES_MASTERS, live_shared_master_themes
shared = live_shared_master_themes(self._package_map())
if shared:
print(f"FAILED - Found {len(shared)} master(s) sharing a theme part:")
for message in shared:
print(f" {message}")
if any(m.startswith(_NOTES_MASTERS) for m in shared):
print(" Fix: in ppt/presentation.xml, move <p:notesMasterIdLst> back to "
"directly after <p:sldIdLst>. PowerPoint reads that happily.")
else:
print(" Fix: give each master its own theme part.")
return False
if self.verbose:
print("PASSED - No master shares a theme part in a way PowerPoint refuses")
return True
def validate_charts(self):
from helpers.pptx_chart import find_chart_problems
problems = find_chart_problems(self._package_map())
if problems:
print(f"FAILED - Found {len(problems)} chart problem(s) PowerPoint rejects:")
for message in problems:
print(f" {message}")
return False
if self.verbose:
print("PASSED - Charts satisfy the constraints PowerPoint enforces")
return True
def _original_slide_defects(self, schema) -> set[str]:
import tempfile
import zipfile
from helpers.pptx_slide import SLIDE_PART_RE, fatal_slide_errors
if self.original_file is None:
return set()
found: set[str] = set()
with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir)
try:
with zipfile.ZipFile(self.original_file, "r") as zf:
safe_extract(zf, temp_path)
except (zipfile.BadZipFile, ValueError, OSError):
return set()
for part in sorted(temp_path.rglob("*.xml")):
relative = part.relative_to(temp_path).as_posix()
if not SLIDE_PART_RE.fullmatch(relative):
continue
ok, errors = self._validate_single_file_xsd(
part.resolve(), temp_path.resolve(), schema_path=schema
)
if ok is None or ok or not errors:
continue
found |= set(fatal_slide_errors(set(errors)))
return found
def validate_slides(self):
from helpers.pptx_slide import (
SLIDE_PART_RE,
fatal_slide_errors,
is_schema_verdict,
)
schema = self.schemas_dir / self.SCHEMA_MAPPINGS["ppt"]
inherited = self._original_slide_defects(schema)
problems: list[str] = []
broken: list[str] = []
for xml_file in self.xml_files:
relative = xml_file.relative_to(self.unpacked_dir).as_posix()
if not SLIDE_PART_RE.fullmatch(relative):
continue
ok, errors = self._validate_single_file_xsd(
xml_file.resolve(), self.unpacked_dir.resolve(), schema_path=schema
)
if ok is None or not errors:
continue
unreadable = [f"{relative}: {e}" for e in errors if not is_schema_verdict(e)]
if unreadable:
broken.extend(unreadable)
continue
if ok:
continue
for message in fatal_slide_errors(set(errors)):
if message in inherited:
continue
problems.append(f"{relative}: {message}")
if broken:
print(f"FAILED - Could not check {len(broken)} slide part(s):")
for message in sorted(broken):
print(f" {message[:240]}")
if problems:
print(f"FAILED - Found {len(problems)} slide problem(s) PowerPoint rejects:")
for message in sorted(problems):
print(f" {message[:240]}")
if broken or problems:
return False
if self.verbose:
print("PASSED - Slide XML has none of the defects PowerPoint refuses")
return True
def _get_schema_path(self, xml_file):
if xml_file.parent.name == "charts" and xml_file.name.startswith("chart"):
return None
return super()._get_schema_path(xml_file)
def _preprocess_for_schema(self, xml_doc, relative_path):
if relative_path.as_posix() != "ppt/presentation.xml":
return xml_doc
root = xml_doc.getroot()
ns = f"{{{self.PRESENTATIONML_NAMESPACE}}}"
notes = root.find(f"{ns}notesMasterIdLst")
slides = root.find(f"{ns}sldIdLst")
if notes is None or slides is None:
return xml_doc
children = list(root)
if children.index(notes) < children.index(slides):
return xml_doc
root.remove(notes)
root.insert(list(root).index(slides), notes)
return xml_doc
def validate_uuid_ids(self): def validate_uuid_ids(self):
import lxml.etree import lxml.etree
@@ -229,17 +395,17 @@ class PPTXSchemaValidator(BaseSchemaValidator):
): ):
rel_type = rel.get("Type", "") rel_type = rel.get("Type", "")
if "notesSlide" in rel_type: if "notesSlide" in rel_type:
target = rel.get("Target", "") part = opc_target(
if target: rel.get("Target", ""),
normalized_target = target.replace("../", "") rels_source_part(rels_file, self.unpacked_dir),
rel.get("TargetMode", ""),
)
if part:
slide_name = rels_file.stem.replace( slide_name = rels_file.stem.replace(
".xml", "" ".xml", ""
) )
if normalized_target not in notes_slide_references: notes_slide_references.setdefault(part, []).append(
notes_slide_references[normalized_target] = []
notes_slide_references[normalized_target].append(
(slide_name, rels_file) (slide_name, rels_file)
) )
skills/pptx/scripts/office/validators/redlining.py
+96 -44
View File
@@ -1,5 +1,14 @@
""" """
Validator for tracked changes in Word documents. Validator for tracked changes in Word documents.
Detects untracked edits in word/document.xml: text that differs from the
original without a <w:ins>/<w:del> wrapper recording it. The tracked changes
that are new relative to the original are undone, and the result is compared
against the original; whatever text still differs was edited without being
tracked.
Only the document body is compared. Headers, footers, footnotes and endnotes
are separate parts and are not checked.
""" """
import subprocess import subprocess
@@ -7,14 +16,18 @@ import tempfile
import zipfile import zipfile
from pathlib import Path from pathlib import Path
import defusedxml.ElementTree as ET
from defusedxml.common import DefusedXmlException
from helpers import rendered_text, safe_extract
class RedliningValidator: class RedliningValidator:
def __init__(self, unpacked_dir, original_docx, verbose=False, author="Claude"): def __init__(self, unpacked_dir, original_docx, verbose=False):
self.unpacked_dir = Path(unpacked_dir) self.unpacked_dir = Path(unpacked_dir)
self.original_docx = Path(original_docx) self.original_docx = Path(original_docx)
self.verbose = verbose self.verbose = verbose
self.author = author
self.namespaces = { self.namespaces = {
"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main" "w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
} }
@@ -28,40 +41,12 @@ class RedliningValidator:
print(f"FAILED - Modified document.xml not found at {modified_file}") print(f"FAILED - Modified document.xml not found at {modified_file}")
return False return False
try:
import xml.etree.ElementTree as ET
tree = ET.parse(modified_file)
root = tree.getroot()
del_elements = root.findall(".//w:del", self.namespaces)
ins_elements = root.findall(".//w:ins", self.namespaces)
author_del_elements = [
elem
for elem in del_elements
if elem.get(f"{{{self.namespaces['w']}}}author") == self.author
]
author_ins_elements = [
elem
for elem in ins_elements
if elem.get(f"{{{self.namespaces['w']}}}author") == self.author
]
if not author_del_elements and not author_ins_elements:
if self.verbose:
print(f"PASSED - No tracked changes by {self.author} found.")
return True
except Exception:
pass
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir) temp_path = Path(temp_dir)
try: try:
with zipfile.ZipFile(self.original_docx, "r") as zip_ref: with zipfile.ZipFile(self.original_docx, "r") as zip_ref:
zip_ref.extractall(temp_path) safe_extract(zip_ref, temp_path)
except Exception as e: except Exception as e:
print(f"FAILED - Error unpacking original docx: {e}") print(f"FAILED - Error unpacking original docx: {e}")
return False return False
@@ -74,18 +59,16 @@ class RedliningValidator:
return False return False
try: try:
import xml.etree.ElementTree as ET
modified_tree = ET.parse(modified_file) modified_tree = ET.parse(modified_file)
modified_root = modified_tree.getroot() modified_root = modified_tree.getroot()
original_tree = ET.parse(original_file) original_tree = ET.parse(original_file)
original_root = original_tree.getroot() original_root = original_tree.getroot()
except ET.ParseError as e: except (ET.ParseError, DefusedXmlException) as e:
print(f"FAILED - Error parsing XML files: {e}") print(f"FAILED - Error parsing XML files: {e}")
return False return False
self._remove_author_tracked_changes(original_root) new_changes = self._new_tracked_changes(original_root, modified_root)
self._remove_author_tracked_changes(modified_root) self._remove_tracked_changes(modified_root, new_changes)
modified_text = self._extract_text_content(modified_root) modified_text = self._extract_text_content(modified_root)
original_text = self._extract_text_content(original_root) original_text = self._extract_text_content(original_root)
@@ -98,20 +81,91 @@ class RedliningValidator:
return False return False
if self.verbose: if self.verbose:
print(f"PASSED - All changes by {self.author} are properly tracked") print(
f"PASSED - All {len(new_changes)} change(s) against the original "
"are properly tracked"
)
return True return True
def _tracked_change_elements(self, root):
ins_tag = f"{{{self.namespaces['w']}}}ins"
del_tag = f"{{{self.namespaces['w']}}}del"
return [elem for elem in root.iter() if elem.tag in (ins_tag, del_tag)]
def _rendered_text(self, elem):
preserve = elem.get("{http://www.w3.org/XML/1998/namespace}space") == "preserve"
return rendered_text(elem.text or "", preserve)
def _text_elements(self, elem):
w = self.namespaces["w"]
return [
node
for node in elem.iter()
if node.tag in (f"{{{w}}}t", f"{{{w}}}delText")
]
def _tracked_change_key(self, elem):
w = self.namespaces["w"]
text = "".join(self._rendered_text(node) for node in self._text_elements(elem))
return (elem.tag, elem.get(f"{{{w}}}author"), elem.get(f"{{{w}}}date"), text)
def _new_tracked_changes(self, original_root, modified_root):
original = self._tracked_change_elements(original_root)
modified = self._tracked_change_elements(modified_root)
pool = {}
for elem in original:
pool.setdefault(self._tracked_change_key(elem), []).append(elem)
matched, leftover = set(), []
for elem in modified:
bucket = pool.get(self._tracked_change_key(elem))
if bucket:
matched.add(bucket.pop())
else:
leftover.append(elem)
def group(elem):
return self._tracked_change_key(elem)[:3]
def text_of(elems):
return "".join(self._tracked_change_key(e)[3] for e in elems)
unmatched_original = {}
for elem in original:
if elem not in matched:
unmatched_original.setdefault(group(elem), []).append(elem)
by_group = {}
for elem in leftover:
by_group.setdefault(group(elem), []).append(elem)
new = set()
for key, elems in by_group.items():
rebuilt = text_of(elems)
if rebuilt and rebuilt == text_of(unmatched_original.get(key, [])):
continue
new.update(elems)
return new
def _generate_detailed_diff(self, original_text, modified_text): def _generate_detailed_diff(self, original_text, modified_text):
error_parts = [ error_parts = [
f"FAILED - Document text doesn't match after removing {self.author}'s tracked changes", "FAILED - Document text doesn't match after removing the tracked changes",
"", "",
"Likely causes:", "Likely causes:",
" 1. Modified text inside another author's <w:ins> or <w:del> tags", " 1. Modified text inside another author's <w:ins> or <w:del> tags",
" 2. Made edits without proper tracked changes", " 2. Made edits without proper tracked changes",
" 3. Didn't nest <w:del> inside <w:ins> when deleting another's insertion", " 3. Didn't nest <w:del> inside <w:ins> when deleting another's insertion",
" 4. Rewrote another author's <w:ins>/<w:del> and changed its text on",
" the way. A tracked change from the original is recognised by its",
" author, date and text; anything that doesn't reproduce one exactly",
" reads as new, and the text it carried is reported missing.",
"", "",
"For pre-redlined documents, use correct patterns:", "For pre-redlined documents, use correct patterns:",
" - To reject another's INSERTION: Nest <w:del> inside their <w:ins>", " - To reject another's INSERTION: Nest <w:del> inside their <w:ins>",
" - To reject PART of one: nest <w:del> around only the runs you reject.",
" Their <w:ins> may be split around it, so long as the pieces keep",
" their author and date and still spell out the same text.",
" - To restore another's DELETION: Add new <w:ins> AFTER their <w:del>", " - To restore another's DELETION: Add new <w:ins> AFTER their <w:del>",
"", "",
] ]
@@ -195,15 +249,14 @@ class RedliningValidator:
return None return None
def _remove_author_tracked_changes(self, root): def _remove_tracked_changes(self, root, targets):
ins_tag = f"{{{self.namespaces['w']}}}ins" ins_tag = f"{{{self.namespaces['w']}}}ins"
del_tag = f"{{{self.namespaces['w']}}}del" del_tag = f"{{{self.namespaces['w']}}}del"
author_attr = f"{{{self.namespaces['w']}}}author"
for parent in root.iter(): for parent in root.iter():
to_remove = [] to_remove = []
for child in parent: for child in parent:
if child.tag == ins_tag and child.get(author_attr) == self.author: if child.tag == ins_tag and child in targets:
to_remove.append(child) to_remove.append(child)
for elem in to_remove: for elem in to_remove:
parent.remove(elem) parent.remove(elem)
@@ -214,7 +267,7 @@ class RedliningValidator:
for parent in root.iter(): for parent in root.iter():
to_process = [] to_process = []
for child in parent: for child in parent:
if child.tag == del_tag and child.get(author_attr) == self.author: if child.tag == del_tag and child in targets:
to_process.append((child, list(parent).index(child))) to_process.append((child, list(parent).index(child)))
for del_elem, del_index in reversed(to_process): for del_elem, del_index in reversed(to_process):
@@ -234,8 +287,7 @@ class RedliningValidator:
for p_elem in root.findall(f".//{p_tag}"): for p_elem in root.findall(f".//{p_tag}"):
text_parts = [] text_parts = []
for t_elem in p_elem.findall(f".//{t_tag}"): for t_elem in p_elem.findall(f".//{t_tag}"):
if t_elem.text: text_parts.append(self._rendered_text(t_elem))
text_parts.append(t_elem.text)
paragraph_text = "".join(text_parts) paragraph_text = "".join(text_parts)
if paragraph_text: if paragraph_text:
paragraphs.append(paragraph_text) paragraphs.append(paragraph_text)
skills/pptx/scripts/thumbnail.py
+49 -27
View File
@@ -16,6 +16,7 @@ Examples:
""" """
import argparse import argparse
import posixpath
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
@@ -23,9 +24,12 @@ import zipfile
from pathlib import Path from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
from office.soffice import get_soffice_env from defusedxml import ElementTree
from office.helpers import SLIDE_REL_TYPE, opc_target
from office.soffice import run_soffice
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
THUMBNAIL_WIDTH = 300 THUMBNAIL_WIDTH = 300
CONVERSION_DPI = 100 CONVERSION_DPI = 100
MAX_COLS = 6 MAX_COLS = 6
@@ -92,28 +96,45 @@ def main():
sys.exit(1) sys.exit(1)
def _is_hidden(zf: zipfile.ZipFile, part: str) -> bool:
try:
with zf.open(part) as f:
for _, root in ElementTree.iterparse(f, events=("start",)):
return root.get("show") in ("0", "false")
except (KeyError, ElementTree.ParseError):
return False
return False
def get_slide_info(pptx_path: Path) -> list[dict]: def get_slide_info(pptx_path: Path) -> list[dict]:
with zipfile.ZipFile(pptx_path, "r") as zf: with zipfile.ZipFile(pptx_path, "r") as zf:
rels_content = zf.read("ppt/_rels/presentation.xml.rels").decode("utf-8") rels_content = zf.read("ppt/_rels/presentation.xml.rels").decode("utf-8")
rels_dom = defusedxml.minidom.parseString(rels_content) rels_dom = defusedxml.minidom.parseString(rels_content)
rid_to_slide = {} rid_to_part = {}
for rel in rels_dom.getElementsByTagName("Relationship"): for rel in rels_dom.getElementsByTagName("Relationship"):
rid = rel.getAttribute("Id") if rel.getAttribute("Type") != SLIDE_REL_TYPE:
target = rel.getAttribute("Target") continue
rel_type = rel.getAttribute("Type") part = opc_target(
if "slide" in rel_type and target.startswith("slides/"): rel.getAttribute("Target"),
rid_to_slide[rid] = target.replace("slides/", "") "ppt/presentation.xml",
rel.getAttribute("TargetMode"),
)
if part is not None:
rid_to_part[rel.getAttribute("Id")] = part
pres_content = zf.read("ppt/presentation.xml").decode("utf-8") pres_content = zf.read("ppt/presentation.xml").decode("utf-8")
pres_dom = defusedxml.minidom.parseString(pres_content) pres_dom = defusedxml.minidom.parseString(pres_content)
present = set(zf.namelist())
slides = [] slides = []
for sld_id in pres_dom.getElementsByTagName("p:sldId"): for sld_id in pres_dom.getElementsByTagName("p:sldId"):
rid = sld_id.getAttribute("r:id") part = rid_to_part.get(sld_id.getAttribute("r:id"))
if rid in rid_to_slide: if part is not None and part in present:
hidden = sld_id.getAttribute("show") == "0" slides.append(
slides.append({"name": rid_to_slide[rid], "hidden": hidden}) {"name": posixpath.basename(part), "hidden": _is_hidden(zf, part)}
)
return slides return slides
@@ -123,6 +144,15 @@ def build_slide_list(
visible_images: list[Path], visible_images: list[Path],
temp_dir: Path, temp_dir: Path,
) -> list[tuple[Path, str]]: ) -> list[tuple[Path, str]]:
visible_count = sum(1 for info in slide_info if not info["hidden"])
rendered_hidden = len(visible_images) == len(slide_info) != visible_count
if not rendered_hidden and visible_count != len(visible_images):
raise ValueError(
f"LibreOffice rendered {len(visible_images)} page(s) for {visible_count} "
f"visible slide(s) of {len(slide_info)}; thumbnails would be mislabeled"
)
if visible_images: if visible_images:
with Image.open(visible_images[0]) as img: with Image.open(visible_images[0]) as img:
placeholder_size = img.size placeholder_size = img.size
@@ -133,15 +163,15 @@ def build_slide_list(
visible_idx = 0 visible_idx = 0
for info in slide_info: for info in slide_info:
if info["hidden"]: if info["hidden"] and not rendered_hidden:
placeholder_path = temp_dir / f"hidden-{info['name']}.jpg" placeholder_path = temp_dir / f"hidden-{info['name']}.jpg"
placeholder_img = create_hidden_placeholder(placeholder_size) placeholder_img = create_hidden_placeholder(placeholder_size)
placeholder_img.save(placeholder_path, "JPEG") placeholder_img.save(placeholder_path, "JPEG")
slides.append((placeholder_path, f"{info['name']} (hidden)")) slides.append((placeholder_path, f"{info['name']} (hidden)"))
else: else:
if visible_idx < len(visible_images): label = f"{info['name']} (hidden)" if info["hidden"] else info["name"]
slides.append((visible_images[visible_idx], info["name"])) slides.append((visible_images[visible_idx], label))
visible_idx += 1 visible_idx += 1
return slides return slides
@@ -158,22 +188,14 @@ def create_hidden_placeholder(size: tuple[int, int]) -> Image.Image:
def convert_to_images(pptx_path: Path, temp_dir: Path) -> list[Path]: def convert_to_images(pptx_path: Path, temp_dir: Path) -> list[Path]:
pdf_path = temp_dir / f"{pptx_path.stem}.pdf" pdf_path = temp_dir / f"{pptx_path.stem}.pdf"
result = subprocess.run( result = run_soffice(
[ ["--headless", "--convert-to", "pdf", "--outdir", str(temp_dir), str(pptx_path)],
"soffice",
"--headless",
"--convert-to",
"pdf",
"--outdir",
str(temp_dir),
str(pptx_path),
],
capture_output=True, capture_output=True,
text=True, text=True,
env=get_soffice_env(),
) )
if result.returncode != 0 or not pdf_path.exists(): if result.returncode != 0 or not pdf_path.exists():
raise RuntimeError("PDF conversion failed") detail = (result.stderr or result.stdout or "").strip()
raise RuntimeError(f"PDF conversion failed: {detail}" if detail else "PDF conversion failed")
result = subprocess.run( result = subprocess.run(
[ [
skills/xlsx/SKILL.md
+82 -275
View File
@@ -1,292 +1,99 @@
--- ---
name: xlsx name: xlsx
description: "Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like \"the xlsx in my downloads\") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved." description: "Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like \"the xlsx in my downloads\") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved."
license: Proprietary. LICENSE.txt has complete terms license: Proprietary. LICENSE.txt has complete terms
--- ---
# Requirements for Outputs # XLSX creation, editing, and analysis
## All Excel files | Task | Approach |
|---|---|
| **Create** or **edit** with formulas/formatting | `openpyxl` — see gotchas below |
| **Bulk data** in or out | `pandas` (`read_excel`, `to_excel`) |
| **Quick look** at a sheet | `markitdown file.xlsx` — `## SheetName` per sheet; reads `.xlsm` too. No cell coordinates, so don't plan edits from it |
| **Read** a model (formulas *and* values) | two `load_workbook` passes — see gotchas |
### Professional Font > `openpyxl`, `pandas`, and `markitdown` are preinstalled — do not run `pip install` first; write the script and import directly. Only if an import fails (or the `markitdown` command is missing): `pip install` the missing package.
- Use a consistent, professional font (e.g., Arial, Times New Roman) for all deliverables unless otherwise instructed by the user
### Zero Formula Errors > Script paths below are relative to this skill's directory.
- Every Excel model MUST be delivered with ZERO formula errors (#REF!, #DIV/0!, #VALUE!, #N/A, #NAME?)
### Preserve Existing Templates (when updating templates) ## Requirements for every output
- Study and EXACTLY match existing format, style, and conventions when modifying files
- Never impose standardized formatting on files with established patterns - **Professional font** (Arial, Times New Roman) throughout, unless the user says otherwise.
- Existing template conventions ALWAYS override these guidelines - **Zero formula errors.** Never ship while `recalc.py` reports `errors_found`. If you think an error predates you, prove it: load the *original* with `data_only=True` and look at that cell. An error you introduced looks exactly like one you inherited.
- **Use formulas, never hardcoded results.** Write `sheet['B10'] = '=SUM(B2:B9)'`, not the Python-computed total. The sheet must recalculate when its inputs change.
- **Follow the user's spec literally.** Exact tab names, exact column headers, and the formula they spelled out. A redesign that computes something else fails, however elegant.
- **Document every assumption and hardcoded number** where the reader will see it — a cell comment, or an adjacent cell at a table's end. Cite a real source when one exists (`Source: Company 10-K, FY2024, Page 45, Revenue Note, [SEC EDGAR URL]`); when the number came from the user, say so plainly.
- **A workbook *you create* for someone to fill in** needs a short legend naming which cells to edit, and one example row of realistic values showing the expected format. Never add such a row to a file you were asked to edit.
- **Editing an existing file: match its conventions exactly.** They override every guideline here. Find its designated input cells first — a distinct font color, fill, or shading marks them — write only there, and leave every existing formula untouched.
## Recalculate (mandatory whenever the file contains formulas)
openpyxl writes formulas as strings with **no cached values**. Until you recalculate, every
formula cell reads back as `None` to anything reading cached values — `pandas`,
`load_workbook(data_only=True)`, and most previewers.
```bash
python scripts/recalc.py output.xlsx [timeout_seconds] # default 30
```
LibreOffice computes every formula, the file is **rewritten in place**, and you get JSON:
`status` (`success` | `errors_found`), `total_formulas`, `total_errors`, and an
`error_summary` naming up to 100 cells per error type (`locations_truncated` says how many it
withheld — trust `total_errors`, not the length of the list). Fix what it names and run it
again. **JSON with an `error` key instead of a `status` means nothing was recalculated**, and
only that case exits non-zero — `errors_found` exits 0, so never treat a clean exit as a clean
workbook.
**A green recalc proves your formulas *evaluate*, not that they are *right*.** An off-by-one
range or a reference to the wrong row yields a clean, error-free file with wrong numbers.
Write 2–3 formulas first and check they pull the values you expect, before building out a grid.
**A workbook that links to another file loses those links** if you re-save it with openpyxl and
then recalculate. Such a formula reads `='[1]Returns Analysis'!$B$2` — the `[1]` is an index
into the workbook's external-reference list, naming a *separate file on disk*, not a sheet.
That file is rarely present here, so the cell's cached value is the only thing holding its
data. openpyxl strips that value on save; LibreOffice then has to resolve the reference for
real, fails, writes `#NAME?`, and deletes every link. `recalc.py` refuses to run in that state
— copy those cells' values out of the original before you save over them (`--force` overrides,
and accepts the loss).
## Choosing formulas that survive verification
LibreOffice implements fewer functions than Excel, and one it cannot evaluate becomes a
literal `#NAME?` baked into the file you deliver.
- **Prefer Excel-2007-era functions** — `SUMIFS`, `INDEX`, `MATCH`, `IFERROR`, `SUMPRODUCT` — which need no prefix.
- **Six post-2007 functions work, but only with an `_xlfn.` prefix**, because openpyxl writes your formula into the XML verbatim and Excel stores post-2007 names prefixed (its UI hides the prefix): `_xlfn.TEXTJOIN`, `_xlfn.CONCAT`, `_xlfn.IFS`, `_xlfn.SWITCH`, `_xlfn.MAXIFS`, `_xlfn.MINIFS`. Written bare, each yields `#NAME?`.
- **Never use `XLOOKUP`, `XMATCH`, `SORT`, `FILTER`, `UNIQUE`, or `SEQUENCE`.** The runtime's LibreOffice cannot evaluate them under *any* prefix. Newer builds do evaluate them, but they are spilling array functions and an openpyxl-written file has no spill metadata, so only the top-left cell of the range gets a value — and `recalc.py` reports `total_errors: 0` on the truncated result. Use `INDEX`/`MATCH` for lookups, and sort, filter, and de-duplicate in Python before writing the cells.
- A formula LibreOffice could not parse is written back **lowercased** — a quick tell beside a `#NAME?`.
## openpyxl gotchas
- **Reading a model takes two loads.** `data_only=True` yields cached values with the formulas gone; the default yields formula strings with no values. One pass cannot give you both.
- **`data_only=True` is destructive if you save.** That workbook has no formulas left, so saving replaces every one with a literal — permanently.
- **`data_only=True` on a file openpyxl just wrote returns `None` everywhere** — run `recalc.py` first. (A formula whose result is `""` also reads back as `None`.)
- **Merged cells: write the top-left anchor only.** Every other cell in the range is a `MergedCell` whose `.value` is read-only.
- **`.xlsm` loses its macros unless you pass `keep_vba=True`** to `load_workbook`.
- **A sheet name containing a space must be quoted** in a cross-sheet reference: `='Assumptions Inputs'!$B$5`. Unquoted, it evaluates to `#VALUE!`.
## Financial models ## Financial models
### Color Coding Standards Unless the user says otherwise, or the existing file already does something else.
Unless otherwise stated by the user or existing template
#### Industry-Standard Color Conventions **Color:** blue text (`0,0,255`) for hardcoded inputs and scenario levers · black for formulas ·
- **Blue text (RGB: 0,0,255)**: Hardcoded inputs, and numbers users will change for scenarios green (`0,128,0`) for links to another sheet · red (`255,0,0`) for links to another file ·
- **Black text (RGB: 0,0,0)**: ALL formulas and calculations yellow fill (`255,255,0`) for key assumptions and cells the user should fill in.
- **Green text (RGB: 0,128,0)**: Links pulling from other worksheets within same workbook
- **Red text (RGB: 255,0,0)**: External links to other files
- **Yellow background (RGB: 255,255,0)**: Key assumptions needing attention or cells that need to be updated
### Number Formatting Standards **Numbers:** currency `$#,##0`, with the unit named in the header (`Revenue ($mm)`) · zeros
render as `-`, including in percentages (`$#,##0;($#,##0);-`) · negatives in parentheses ·
percentages `0.0%`, **stored as fractions** (`0.15` renders `15.0%`; storing `15` renders
`1500.0%`) · valuation multiples `0.0x` · years as text (`"2024"`, never `2,024`).
#### Required Format Rules **Structure:** every assumption in its own labeled cell, referenced by the formulas that use it
- **Years**: Format as text strings (e.g., "2024" not "2,024") (`=B5*(1+$B$6)`, never `=B5*1.05`) · formulas consistent across every projection period, since a
- **Currency**: Use $#,##0 format; ALWAYS specify units in headers ("Revenue ($mm)") lone edited cell mid-row is the commonest silent error · guard denominators that can be zero.
- **Zeros**: Use number formatting to make all zeros "-", including percentages (e.g., "$#,##0;($#,##0);-")
- **Percentages**: Default to 0.0% format (one decimal)
- **Multiples**: Format as 0.0x for valuation multiples (EV/EBITDA, P/E)
- **Negative numbers**: Use parentheses (123) not minus -123
### Formula Construction Rules ## Dependencies
#### Assumptions Placement `openpyxl`, `pandas`, `markitdown` (pip, preinstalled — install only if an import fails or the command is missing) · LibreOffice (`soffice`, auto-configured for sandboxed environments via `scripts/office/soffice.py`)
- Place ALL assumptions (growth rates, margins, multiples, etc.) in separate assumption cells
- Use cell references instead of hardcoded values in formulas
- Example: Use =B5*(1+$B$6) instead of =B5*1.05
#### Formula Error Prevention
- Verify all cell references are correct
- Check for off-by-one errors in ranges
- Ensure consistent formulas across all projection periods
- Test with edge cases (zero values, negative numbers)
- Verify no unintended circular references
#### Documentation Requirements for Hardcodes
- Comment or in cells beside (if end of table). Format: "Source: [System/Document], [Date], [Specific Reference], [URL if applicable]"
- Examples:
- "Source: Company 10-K, FY2024, Page 45, Revenue Note, [SEC EDGAR URL]"
- "Source: Company 10-Q, Q2 2025, Exhibit 99.1, [SEC EDGAR URL]"
- "Source: Bloomberg Terminal, 8/15/2025, AAPL US Equity"
- "Source: FactSet, 8/20/2025, Consensus Estimates Screen"
# XLSX creation, editing, and analysis
## Overview
A user may ask you to create, edit, or analyze the contents of an .xlsx file. You have different tools and workflows available for different tasks.
## Important Requirements
**LibreOffice Required for Formula Recalculation**: You can assume LibreOffice is installed for recalculating formula values using the `scripts/recalc.py` script. The script automatically configures LibreOffice on first run, including in sandboxed environments where Unix sockets are restricted (handled by `scripts/office/soffice.py`)
## Reading and analyzing data
### Data analysis with pandas
For data analysis, visualization, and basic operations, use **pandas** which provides powerful data manipulation capabilities:
```python
import pandas as pd
# Read Excel
df = pd.read_excel('file.xlsx') # Default: first sheet
all_sheets = pd.read_excel('file.xlsx', sheet_name=None) # All sheets as dict
# Analyze
df.head() # Preview data
df.info() # Column info
df.describe() # Statistics
# Write Excel
df.to_excel('output.xlsx', index=False)
```
## Excel File Workflows
## CRITICAL: Use Formulas, Not Hardcoded Values
**Always use Excel formulas instead of calculating values in Python and hardcoding them.** This ensures the spreadsheet remains dynamic and updateable.
### ❌ WRONG - Hardcoding Calculated Values
```python
# Bad: Calculating in Python and hardcoding result
total = df['Sales'].sum()
sheet['B10'] = total # Hardcodes 5000
# Bad: Computing growth rate in Python
growth = (df.iloc[-1]['Revenue'] - df.iloc[0]['Revenue']) / df.iloc[0]['Revenue']
sheet['C5'] = growth # Hardcodes 0.15
# Bad: Python calculation for average
avg = sum(values) / len(values)
sheet['D20'] = avg # Hardcodes 42.5
```
### ✅ CORRECT - Using Excel Formulas
```python
# Good: Let Excel calculate the sum
sheet['B10'] = '=SUM(B2:B9)'
# Good: Growth rate as Excel formula
sheet['C5'] = '=(C4-C2)/C2'
# Good: Average using Excel function
sheet['D20'] = '=AVERAGE(D2:D19)'
```
This applies to ALL calculations - totals, percentages, ratios, differences, etc. The spreadsheet should be able to recalculate when source data changes.
## Common Workflow
1. **Choose tool**: pandas for data, openpyxl for formulas/formatting
2. **Create/Load**: Create new workbook or load existing file
3. **Modify**: Add/edit data, formulas, and formatting
4. **Save**: Write to file
5. **Recalculate formulas (MANDATORY IF USING FORMULAS)**: Use the scripts/recalc.py script
```bash
python scripts/recalc.py output.xlsx
```
6. **Verify and fix any errors**:
- The script returns JSON with error details
- If `status` is `errors_found`, check `error_summary` for specific error types and locations
- Fix the identified errors and recalculate again
- Common errors to fix:
- `#REF!`: Invalid cell references
- `#DIV/0!`: Division by zero
- `#VALUE!`: Wrong data type in formula
- `#NAME?`: Unrecognized formula name
### Creating new Excel files
```python
# Using openpyxl for formulas and formatting
from openpyxl import Workbook
from openpyxl.styles import Font, PatternFill, Alignment
wb = Workbook()
sheet = wb.active
# Add data
sheet['A1'] = 'Hello'
sheet['B1'] = 'World'
sheet.append(['Row', 'of', 'data'])
# Add formula
sheet['B2'] = '=SUM(A1:A10)'
# Formatting
sheet['A1'].font = Font(bold=True, color='FF0000')
sheet['A1'].fill = PatternFill('solid', start_color='FFFF00')
sheet['A1'].alignment = Alignment(horizontal='center')
# Column width
sheet.column_dimensions['A'].width = 20
wb.save('output.xlsx')
```
### Editing existing Excel files
```python
# Using openpyxl to preserve formulas and formatting
from openpyxl import load_workbook
# Load existing file
wb = load_workbook('existing.xlsx')
sheet = wb.active # or wb['SheetName'] for specific sheet
# Working with multiple sheets
for sheet_name in wb.sheetnames:
sheet = wb[sheet_name]
print(f"Sheet: {sheet_name}")
# Modify cells
sheet['A1'] = 'New Value'
sheet.insert_rows(2) # Insert row at position 2
sheet.delete_cols(3) # Delete column 3
# Add new sheet
new_sheet = wb.create_sheet('NewSheet')
new_sheet['A1'] = 'Data'
wb.save('modified.xlsx')
```
## Recalculating formulas
Excel files created or modified by openpyxl contain formulas as strings but not calculated values. Use the provided `scripts/recalc.py` script to recalculate formulas:
```bash
python scripts/recalc.py <excel_file> [timeout_seconds]
```
Example:
```bash
python scripts/recalc.py output.xlsx 30
```
The script:
- Automatically sets up LibreOffice macro on first run
- Recalculates all formulas in all sheets
- Scans ALL cells for Excel errors (#REF!, #DIV/0!, etc.)
- Returns JSON with detailed error locations and counts
- Works on both Linux and macOS
## Formula Verification Checklist
Quick checks to ensure formulas work correctly:
### Essential Verification
- [ ] **Test 2-3 sample references**: Verify they pull correct values before building full model
- [ ] **Column mapping**: Confirm Excel columns match (e.g., column 64 = BL, not BK)
- [ ] **Row offset**: Remember Excel rows are 1-indexed (DataFrame row 5 = Excel row 6)
### Common Pitfalls
- [ ] **NaN handling**: Check for null values with `pd.notna()`
- [ ] **Far-right columns**: FY data often in columns 50+
- [ ] **Multiple matches**: Search all occurrences, not just first
- [ ] **Division by zero**: Check denominators before using `/` in formulas (#DIV/0!)
- [ ] **Wrong references**: Verify all cell references point to intended cells (#REF!)
- [ ] **Cross-sheet references**: Use correct format (Sheet1!A1) for linking sheets
### Formula Testing Strategy
- [ ] **Start small**: Test formulas on 2-3 cells before applying broadly
- [ ] **Verify dependencies**: Check all cells referenced in formulas exist
- [ ] **Test edge cases**: Include zero, negative, and very large values
### Interpreting scripts/recalc.py Output
The script returns JSON with error details:
```json
{
"status": "success", // or "errors_found"
"total_errors": 0, // Total error count
"total_formulas": 42, // Number of formulas in file
"error_summary": { // Only present if errors found
"#REF!": {
"count": 2,
"locations": ["Sheet1!B5", "Sheet1!C10"]
}
}
}
```
## Best Practices
### Library Selection
- **pandas**: Best for data analysis, bulk operations, and simple data export
- **openpyxl**: Best for complex formatting, formulas, and Excel-specific features
### Working with openpyxl
- Cell indices are 1-based (row=1, column=1 refers to cell A1)
- Use `data_only=True` to read calculated values: `load_workbook('file.xlsx', data_only=True)`
- **Warning**: If opened with `data_only=True` and saved, formulas are replaced with values and permanently lost
- For large files: Use `read_only=True` for reading or `write_only=True` for writing
- Formulas are preserved but not evaluated - use scripts/recalc.py to update values
### Working with pandas
- Specify data types to avoid inference issues: `pd.read_excel('file.xlsx', dtype={'id': str})`
- For large files, read specific columns: `pd.read_excel('file.xlsx', usecols=['A', 'C', 'E'])`
- Handle dates properly: `pd.read_excel('file.xlsx', parse_dates=['date_column'])`
## Code Style Guidelines
**IMPORTANT**: When generating Python code for Excel operations:
- Write minimal, concise Python code without unnecessary comments
- Avoid verbose variable names and redundant operations
- Avoid unnecessary print statements
**For Excel files themselves**:
- Add comments to cells with complex formulas or important assumptions
- Document data sources for hardcoded values
- Include notes for key calculations and model sections
skills/xlsx/scripts/office/helpers/__init__.py
+111
View File
@@ -0,0 +1,111 @@
import os
import posixpath
import re
import stat
import tempfile
import urllib.parse
import zipfile
from pathlib import Path
OOXML_FAMILY = {
".docx": "docx",
".dotx": "docx",
".pptx": "pptx",
".potx": "pptx",
".xlsx": "xlsx",
".xltx": "xlsx",
}
_SCHEME_RE = re.compile(r"^[A-Za-z][A-Za-z0-9+.\-]*:")
SLIDE_REL_TYPE = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/slide"
def opc_target(target: str, source_part: str, target_mode: str = "") -> str | None:
if not target:
return None
if target_mode.lower() == "external":
return None
if _SCHEME_RE.match(target):
return None
target = urllib.parse.unquote(target)
if "\\" in target:
raise ValueError(f"relationship target is not a POSIX part name: {target!r}")
if target.startswith("/"):
joined = target.lstrip("/")
else:
joined = posixpath.join(posixpath.dirname(source_part), target)
parts: list[str] = []
for segment in posixpath.normpath(joined).split("/"):
if segment in ("", "."):
continue
if segment == "..":
if not parts:
raise ValueError(f"relationship target escapes the package: {target!r}")
parts.pop()
else:
parts.append(segment)
if not parts:
raise ValueError(f"relationship target resolves to nothing: {target!r}")
return "/".join(parts)
def rels_source_part(rels_file: Path, unpacked_dir: Path) -> str:
owner_dir = rels_file.parent.parent.relative_to(unpacked_dir)
return posixpath.join(owner_dir.as_posix(), rels_file.name[: -len(".rels")]).lstrip("./")
def part_text(data: bytes) -> str:
return data.decode("utf-8", "surrogateescape")
XML_SPACE = " \t\r\n"
def rendered_text(text: str, preserve: bool) -> str:
return text if preserve else text.strip(XML_SPACE)
def safe_extract(zf: zipfile.ZipFile, dest: Path) -> None:
dest = dest.resolve()
for m in zf.infolist():
if stat.S_ISLNK(m.external_attr >> 16):
raise ValueError(f"symlink archive entry not allowed: {m.filename!r}")
target = (dest / m.filename).resolve()
if not target.is_relative_to(dest):
raise ValueError(f"unsafe archive entry: {m.filename!r}")
zf.extract(m, dest)
def rezip(src_dir: Path, out_path: Path) -> None:
files = sorted(p for p in src_dir.rglob("*") if p.is_file())
ct = src_dir / "[Content_Types].xml"
fd, tmp_name = tempfile.mkstemp(
prefix=out_path.name + ".", suffix=".tmp", dir=out_path.parent
)
tmp_out = Path(tmp_name)
try:
with os.fdopen(fd, "wb") as fh:
with zipfile.ZipFile(fh, "w", zipfile.ZIP_DEFLATED) as zf:
if ct.exists():
zf.write(ct, ct.relative_to(src_dir), compress_type=zipfile.ZIP_STORED)
for f in files:
if f == ct:
continue
zf.write(f, f.relative_to(src_dir))
if out_path.exists():
mode = out_path.stat().st_mode & 0o777
else:
umask = os.umask(0)
os.umask(umask)
mode = 0o666 & ~umask
os.chmod(tmp_out, mode)
os.replace(tmp_out, out_path)
finally:
if tmp_out.exists():
tmp_out.unlink()
skills/xlsx/scripts/office/helpers/merge_runs.py
-199
View File
@@ -1,199 +0,0 @@
"""Merge adjacent runs with identical formatting in DOCX.
Merges adjacent <w:r> elements that have identical <w:rPr> properties.
Works on runs in paragraphs and inside tracked changes (<w:ins>, <w:del>).
Also:
- Removes rsid attributes from runs (revision metadata that doesn't affect rendering)
- Removes proofErr elements (spell/grammar markers that block merging)
"""
from pathlib import Path
import defusedxml.minidom
def merge_runs(input_dir: str) -> tuple[int, str]:
doc_xml = Path(input_dir) / "word" / "document.xml"
if not doc_xml.exists():
return 0, f"Error: {doc_xml} not found"
try:
dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8"))
root = dom.documentElement
_remove_elements(root, "proofErr")
_strip_run_rsid_attrs(root)
containers = {run.parentNode for run in _find_elements(root, "r")}
merge_count = 0
for container in containers:
merge_count += _merge_runs_in(container)
doc_xml.write_bytes(dom.toxml(encoding="UTF-8"))
return merge_count, f"Merged {merge_count} runs"
except Exception as e:
return 0, f"Error: {e}"
def _find_elements(root, tag: str) -> list:
results = []
def traverse(node):
if node.nodeType == node.ELEMENT_NODE:
name = node.localName or node.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(node)
for child in node.childNodes:
traverse(child)
traverse(root)
return results
def _get_child(parent, tag: str):
for child in parent.childNodes:
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name == tag or name.endswith(f":{tag}"):
return child
return None
def _get_children(parent, tag: str) -> list:
results = []
for child in parent.childNodes:
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(child)
return results
def _is_adjacent(elem1, elem2) -> bool:
node = elem1.nextSibling
while node:
if node == elem2:
return True
if node.nodeType == node.ELEMENT_NODE:
return False
if node.nodeType == node.TEXT_NODE and node.data.strip():
return False
node = node.nextSibling
return False
def _remove_elements(root, tag: str):
for elem in _find_elements(root, tag):
if elem.parentNode:
elem.parentNode.removeChild(elem)
def _strip_run_rsid_attrs(root):
for run in _find_elements(root, "r"):
for attr in list(run.attributes.values()):
if "rsid" in attr.name.lower():
run.removeAttribute(attr.name)
def _merge_runs_in(container) -> int:
merge_count = 0
run = _first_child_run(container)
while run:
while True:
next_elem = _next_element_sibling(run)
if next_elem and _is_run(next_elem) and _can_merge(run, next_elem):
_merge_run_content(run, next_elem)
container.removeChild(next_elem)
merge_count += 1
else:
break
_consolidate_text(run)
run = _next_sibling_run(run)
return merge_count
def _first_child_run(container):
for child in container.childNodes:
if child.nodeType == child.ELEMENT_NODE and _is_run(child):
return child
return None
def _next_element_sibling(node):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
return sibling
sibling = sibling.nextSibling
return None
def _next_sibling_run(node):
sibling = node.nextSibling
while sibling:
if sibling.nodeType == sibling.ELEMENT_NODE:
if _is_run(sibling):
return sibling
sibling = sibling.nextSibling
return None
def _is_run(node) -> bool:
name = node.localName or node.tagName
return name == "r" or name.endswith(":r")
def _can_merge(run1, run2) -> bool:
rpr1 = _get_child(run1, "rPr")
rpr2 = _get_child(run2, "rPr")
if (rpr1 is None) != (rpr2 is None):
return False
if rpr1 is None:
return True
return rpr1.toxml() == rpr2.toxml()
def _merge_run_content(target, source):
for child in list(source.childNodes):
if child.nodeType == child.ELEMENT_NODE:
name = child.localName or child.tagName
if name != "rPr" and not name.endswith(":rPr"):
target.appendChild(child)
def _consolidate_text(run):
t_elements = _get_children(run, "t")
for i in range(len(t_elements) - 1, 0, -1):
curr, prev = t_elements[i], t_elements[i - 1]
if _is_adjacent(prev, curr):
prev_text = prev.firstChild.data if prev.firstChild else ""
curr_text = curr.firstChild.data if curr.firstChild else ""
merged = prev_text + curr_text
if prev.firstChild:
prev.firstChild.data = merged
else:
prev.appendChild(run.ownerDocument.createTextNode(merged))
if merged.startswith(" ") or merged.endswith(" "):
prev.setAttribute("xml:space", "preserve")
elif prev.hasAttribute("xml:space"):
prev.removeAttribute("xml:space")
run.removeChild(curr)
skills/xlsx/scripts/office/helpers/pptx_chart.py
+170
View File
@@ -0,0 +1,170 @@
"""Find chart XML that PowerPoint refuses but the schema accepts.
Detection only: for either fault more than one repair is valid, and only the
author knows which was meant.
"""
from __future__ import annotations
import re
from typing import Mapping
from . import part_text
_CHART_PART_RE = re.compile(r"ppt/charts/chart\d+\.xml")
_GROUPING_RE = re.compile(r"""<c:grouping\b[^>]*?\bval=["'](\w+)["']""")
_DLBL_POS_RE = re.compile(r"""<c:dLblPos\b[^>]*?\bval=["'](\w+)["']""")
def _strip_ext_lst(text: str) -> str:
out, cursor = [], 0
for lo, hi in _ext_lst_spans(text):
out.append(text[cursor:lo])
cursor = hi
out.append(text[cursor:])
return "".join(out)
_BAR_GROUP_RE = re.compile(r"<c:(bar3DChart|barChart)\b[^>]*(?<!/)>.*?</c:\1\s*>", re.DOTALL)
STACKED_GROUPINGS = frozenset({"stacked", "percentStacked"})
ILLEGAL_ON_STACKED = frozenset({"outEnd"})
LEGAL_ON_STACKED = ("ctr", "inEnd", "inBase")
def _check_stacked_label_positions(part: str, xml: str) -> list[str]:
problems: list[str] = []
for match in _BAR_GROUP_RE.finditer(xml):
block = _strip_ext_lst(match.group(0))
group = match.group(1)
grouping = _GROUPING_RE.search(block)
if grouping is None or grouping.group(1) not in STACKED_GROUPINGS:
continue
bad = [p for p in _DLBL_POS_RE.findall(block) if p in ILLEGAL_ON_STACKED]
for pos in sorted(set(bad)):
problems.append(
f'{part}: {bad.count(pos)} data label(s) use dLblPos="{pos}" on a '
f"{grouping.group(1)} {group}; PowerPoint allows only "
f"{', '.join(LEGAL_ON_STACKED)} there"
)
return problems
_ANY_CHART_GROUP_RE = re.compile(r"<c:(\w+Chart)\b[^>]*(?<!/)>.*?</c:\1\s*>", re.DOTALL)
_AXID_RE = re.compile(
r"""\s*<c:axId\b[^>]*?\bval=["'](-?\d+)["']\s*(?:/>|>\s*</c:axId\s*>)"""
)
_AXIS_DECL_RE = re.compile(
r"""<c:(catAx|valAx|serAx|dateAx)\b[^>]*(?<!/)>\s*<c:axId\b[^>]*?\bval=["'](-?\d+)["']"""
)
AXID_LIMIT = {
"barChart": 2, "lineChart": 2, "areaChart": 2, "scatterChart": 2,
"bubbleChart": 2, "radarChart": 2, "stockChart": 2,
"bar3DChart": 3, "line3DChart": 3, "area3DChart": 3,
"surfaceChart": 3, "surface3DChart": 3,
}
AXID_MINIMUM = {
"barChart": 2, "lineChart": 2, "areaChart": 2, "scatterChart": 2,
"bubbleChart": 2, "radarChart": 2, "stockChart": 2,
"bar3DChart": 2, "area3DChart": 2, "surfaceChart": 2,
"line3DChart": 3, "surface3DChart": 3,
}
def _declared_axes(xml: str) -> dict[str, list[str]]:
axes: dict[str, list[str]] = {}
for kind, axid in _AXIS_DECL_RE.findall(xml):
axes.setdefault(kind, []).append(axid)
return axes
def _canonical_ids(axes: dict[str, list[str]], limit: int) -> list[str] | None:
category = axes.get("catAx", []) + axes.get("dateAx", [])
value = axes.get("valAx", [])
series = axes.get("serAx", [])
if len(category) != 1 or len(value) != 1 or len(series) > 1:
return None
ids = [category[0], value[0]]
if limit >= 3 and series:
ids.append(series[0])
return ids
def _undeclared_axes(kind: str, block: str, axes: dict[str, list[str]]) -> list[str] | None:
if kind not in AXID_LIMIT:
return None
ids = _AXID_RE.findall(block)
declared = {i for group in axes.values() for i in group}
if len([i for i in ids if i in declared]) >= 2:
return None
return ids
def _check_chart_axis_references(part: str, xml: str) -> list[str]:
axes = _declared_axes(xml)
problems: list[str] = []
declared = {i for group in axes.values() for i in group}
for match in _ANY_CHART_GROUP_RE.finditer(xml):
kind, block = match.group(1), match.group(0)
ids = _undeclared_axes(kind, block, axes)
if ids is None:
continue
if not ids:
problems.append(
f"{part}: <c:{kind}> declares no <c:axId> this part can resolve; a chart "
f"group needs {AXID_MINIMUM[kind]}, and PowerPoint discards one with fewer"
)
continue
dead = [i for i in ids if i not in declared]
canonical = _canonical_ids(axes, AXID_LIMIT[kind])
if canonical is not None and len(canonical) >= AXID_MINIMUM[kind]:
hint = f"Fix: point them at the axes this part declares ({', '.join(canonical)})"
else:
hint = ("Fix: the part declares several axes of a kind -- declare the "
"secondary axes the series expects, or drop them")
detail = (f"of which {', '.join(dead)} name no declared axis"
if dead else f"only {len(ids)} of which this part declares")
problems.append(
f"{part}: <c:{kind}> references axId {', '.join(ids)}, {detail}, "
f"leaving fewer than two live axes; PowerPoint discards the chart. {hint}"
)
return problems
def _ext_lst_spans(text: str) -> list[tuple[int, int]]:
spans: list[tuple[int, int]] = []
depth = 0
start = 0
for match in re.finditer(r"<(/?)c:extLst\b[^>]*?(/?)>", text):
closing, self_closing = match.group(1), match.group(2)
if self_closing:
continue
if closing:
depth -= 1
if depth == 0:
spans.append((start, match.end()))
else:
if depth == 0:
start = match.start()
depth += 1
return spans
CHART_CHECKS = (_check_stacked_label_positions, _check_chart_axis_references)
def find_chart_problems(files: Mapping[str, bytes]) -> list[str]:
problems: list[str] = []
for part in sorted(n for n in files if _CHART_PART_RE.fullmatch(n)):
xml = part_text(files[part])
for check in CHART_CHECKS:
problems.extend(check(part, xml))
return problems
skills/xlsx/scripts/office/helpers/pptx_slide.py
+60
View File
@@ -0,0 +1,60 @@
"""Pick the slide-XML schema errors PowerPoint refuses the file over.
A denylist over lxml's messages, so an unrecognised error class is a miss rather
than a false alarm.
"""
from __future__ import annotations
import re
SLIDE_PART_RE = re.compile(
r"ppt/(slides|slideLayouts|slideMasters|notesSlides|notesMasters|handoutMasters)"
r"/[^/]+\.xml"
)
FATAL_SLIDE_ERRORS: tuple[tuple[re.Pattern[str], str], ...] = (
(
re.compile(r"\}tableStyleId': This element is not expected"),
"two <a:tableStyleId> in one <a:tblPr> (the schema allows one)",
),
(
re.compile(r"\}srgbClr', attribute 'val'"),
"a colour that is not six hex digits",
),
(
re.compile(r"\}txBody': Missing child element"),
"a <p:txBody> with no children",
),
(
re.compile(r"\}miter', attribute 'lim'"),
'a line join with lim="NaN"',
),
(
re.compile(r"\}uLnTx': This element is not expected"),
"<a:uLnTx> in a position the schema forbids",
),
(
re.compile(r"\}overrideClrMapping': This element is not expected"),
"<p:overrideClrMapping> in a position the schema forbids",
),
(
re.compile(r"\}nvGrpSpPr': Missing child element"),
"a <p:nvGrpSpPr> with no children",
),
)
def is_schema_verdict(error: str) -> bool:
return error.startswith("Element ")
def fatal_slide_errors(errors: set[str]) -> list[str]:
out = []
for error in sorted(errors):
for pattern, meaning in FATAL_SLIDE_ERRORS:
if pattern.search(error):
out.append(f"{meaning}: {error}")
break
return out
skills/xlsx/scripts/office/helpers/pptx_theme.py
+114
View File
@@ -0,0 +1,114 @@
"""Find masters sharing a theme part in the way PowerPoint refuses to open.
Reports only; the fix is to move <p:notesMasterIdLst> back to directly after
<p:sldIdLst> in ppt/presentation.xml.
"""
from __future__ import annotations
import posixpath
import re
from typing import Mapping
from . import part_text
THEME_REL_TYPE = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/theme"
_MASTER_RE = re.compile(
r"^ppt/(?P<group>slideMasters|notesMasters|handoutMasters)/"
r"(?:slide|notes|handout)Master(?P<num>\d+)\.xml$"
)
_GROUP_ORDER = {"slideMasters": 0, "notesMasters": 1, "handoutMasters": 2}
_RELATIONSHIP_RE = re.compile(
r"<Relationship\b[^>]*?(?:/>|>.*?</Relationship\s*>)", re.DOTALL
)
def _sort_key(name: str) -> tuple[int, int]:
m = _MASTER_RE.match(name)
assert m is not None
return (_GROUP_ORDER[m.group("group")], int(m.group("num")))
def _rels_path(part: str) -> str:
directory, base = posixpath.split(part)
return f"{directory}/_rels/{base}.rels"
def _resolve(rels_path: str, target: str) -> str:
if target.startswith("/"):
return target.lstrip("/")
part_dir = posixpath.dirname(posixpath.dirname(rels_path))
return posixpath.normpath(posixpath.join(part_dir, target))
def _theme_rel(files: Mapping[str, bytes], master: str):
rels_path = _rels_path(master)
rels = files.get(rels_path)
if rels is None:
return None
for element in _RELATIONSHIP_RE.findall(part_text(rels)):
if f'Type="{THEME_REL_TYPE}"' not in element:
continue
target = re.search(r'\bTarget="([^"]+)"', element)
if target is None:
continue
return rels_path, element, _resolve(rels_path, target.group(1))
return None
def _masters(files: Mapping[str, bytes]) -> list[str]:
return sorted((n for n in files if _MASTER_RE.match(n)), key=_sort_key)
_PRESENTATION = "ppt/presentation.xml"
_NOTES_MASTERS = "ppt/notesMasters/"
_IGNORABLE_RE = re.compile(r"<!--.*?-->|<\?.*?\?>", re.DOTALL)
_AFTER_SLDIDLST_RE = re.compile(
r"<p:sldIdLst\b(?:[^>]*/>|[^>]*>.*?</p:sldIdLst\s*>)\s*(<[^>\s/]+)", re.DOTALL
)
def _notes_master_share_is_inert(files: Mapping[str, bytes]) -> bool:
data = files.get(_PRESENTATION)
if data is None:
return False
match = _AFTER_SLDIDLST_RE.search(_IGNORABLE_RE.sub("", part_text(data)))
return match is not None and match.group(1) == "<p:notesMasterIdLst"
def _shares(files: Mapping[str, bytes]):
owner: dict[str, str] = {}
for master in _masters(files):
found = _theme_rel(files, master)
if found is None:
continue
rels_path, element, theme = found
if theme not in files:
continue
if theme in owner:
yield master, rels_path, element, theme, owner[theme]
else:
owner[theme] = master
def _is_inert(master: str, inert_notes: bool) -> bool:
return inert_notes and master.startswith(_NOTES_MASTERS)
def find_shared_master_themes(files: Mapping[str, bytes]) -> list[str]:
return [
f"{master} shares {theme} with {first}"
for master, _, _, theme, first in _shares(files)
]
def live_shared_master_themes(files: Mapping[str, bytes]) -> list[str]:
inert_notes = _notes_master_share_is_inert(files)
return [
f"{master} shares {theme} with {first}"
for master, _, _, theme, first in _shares(files)
if not _is_inert(master, inert_notes)
]
skills/xlsx/scripts/office/helpers/simplify_redlines.py
-197
View File
@@ -1,197 +0,0 @@
"""Simplify tracked changes by merging adjacent w:ins or w:del elements.
Merges adjacent <w:ins> elements from the same author into a single element.
Same for <w:del> elements. This makes heavily-redlined documents easier to
work with by reducing the number of tracked change wrappers.
Rules:
- Only merges w:ins with w:ins, w:del with w:del (same element type)
- Only merges if same author (ignores timestamp differences)
- Only merges if truly adjacent (only whitespace between them)
"""
import xml.etree.ElementTree as ET
import zipfile
from pathlib import Path
import defusedxml.minidom
WORD_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
def simplify_redlines(input_dir: str) -> tuple[int, str]:
doc_xml = Path(input_dir) / "word" / "document.xml"
if not doc_xml.exists():
return 0, f"Error: {doc_xml} not found"
try:
dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8"))
root = dom.documentElement
merge_count = 0
containers = _find_elements(root, "p") + _find_elements(root, "tc")
for container in containers:
merge_count += _merge_tracked_changes_in(container, "ins")
merge_count += _merge_tracked_changes_in(container, "del")
doc_xml.write_bytes(dom.toxml(encoding="UTF-8"))
return merge_count, f"Simplified {merge_count} tracked changes"
except Exception as e:
return 0, f"Error: {e}"
def _merge_tracked_changes_in(container, tag: str) -> int:
merge_count = 0
tracked = [
child
for child in container.childNodes
if child.nodeType == child.ELEMENT_NODE and _is_element(child, tag)
]
if len(tracked) < 2:
return 0
i = 0
while i < len(tracked) - 1:
curr = tracked[i]
next_elem = tracked[i + 1]
if _can_merge_tracked(curr, next_elem):
_merge_tracked_content(curr, next_elem)
container.removeChild(next_elem)
tracked.pop(i + 1)
merge_count += 1
else:
i += 1
return merge_count
def _is_element(node, tag: str) -> bool:
name = node.localName or node.tagName
return name == tag or name.endswith(f":{tag}")
def _get_author(elem) -> str:
author = elem.getAttribute("w:author")
if not author:
for attr in elem.attributes.values():
if attr.localName == "author" or attr.name.endswith(":author"):
return attr.value
return author
def _can_merge_tracked(elem1, elem2) -> bool:
if _get_author(elem1) != _get_author(elem2):
return False
node = elem1.nextSibling
while node and node != elem2:
if node.nodeType == node.ELEMENT_NODE:
return False
if node.nodeType == node.TEXT_NODE and node.data.strip():
return False
node = node.nextSibling
return True
def _merge_tracked_content(target, source):
while source.firstChild:
child = source.firstChild
source.removeChild(child)
target.appendChild(child)
def _find_elements(root, tag: str) -> list:
results = []
def traverse(node):
if node.nodeType == node.ELEMENT_NODE:
name = node.localName or node.tagName
if name == tag or name.endswith(f":{tag}"):
results.append(node)
for child in node.childNodes:
traverse(child)
traverse(root)
return results
def get_tracked_change_authors(doc_xml_path: Path) -> dict[str, int]:
if not doc_xml_path.exists():
return {}
try:
tree = ET.parse(doc_xml_path)
root = tree.getroot()
except ET.ParseError:
return {}
namespaces = {"w": WORD_NS}
author_attr = f"{{{WORD_NS}}}author"
authors: dict[str, int] = {}
for tag in ["ins", "del"]:
for elem in root.findall(f".//w:{tag}", namespaces):
author = elem.get(author_attr)
if author:
authors[author] = authors.get(author, 0) + 1
return authors
def _get_authors_from_docx(docx_path: Path) -> dict[str, int]:
try:
with zipfile.ZipFile(docx_path, "r") as zf:
if "word/document.xml" not in zf.namelist():
return {}
with zf.open("word/document.xml") as f:
tree = ET.parse(f)
root = tree.getroot()
namespaces = {"w": WORD_NS}
author_attr = f"{{{WORD_NS}}}author"
authors: dict[str, int] = {}
for tag in ["ins", "del"]:
for elem in root.findall(f".//w:{tag}", namespaces):
author = elem.get(author_attr)
if author:
authors[author] = authors.get(author, 0) + 1
return authors
except (zipfile.BadZipFile, ET.ParseError):
return {}
def infer_author(modified_dir: Path, original_docx: Path, default: str = "Claude") -> str:
modified_xml = modified_dir / "word" / "document.xml"
modified_authors = get_tracked_change_authors(modified_xml)
if not modified_authors:
return default
original_authors = _get_authors_from_docx(original_docx)
new_changes: dict[str, int] = {}
for author, count in modified_authors.items():
original_count = original_authors.get(author, 0)
diff = count - original_count
if diff > 0:
new_changes[author] = diff
if not new_changes:
return default
if len(new_changes) == 1:
return next(iter(new_changes))
raise ValueError(
f"Multiple authors added new changes: {new_changes}. "
"Cannot infer which author to validate."
)
skills/xlsx/scripts/office/pack.py
-159
View File
@@ -1,159 +0,0 @@
"""Pack a directory into a DOCX, PPTX, or XLSX file.
Validates with auto-repair, condenses XML formatting, and creates the Office file.
Usage:
python pack.py <input_directory> <output_file> [--original <file>] [--validate true|false]
Examples:
python pack.py unpacked/ output.docx --original input.docx
python pack.py unpacked/ output.pptx --validate false
"""
import argparse
import sys
import shutil
import tempfile
import zipfile
from pathlib import Path
import defusedxml.minidom
from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator
def pack(
input_directory: str,
output_file: str,
original_file: str | None = None,
validate: bool = True,
infer_author_func=None,
) -> tuple[None, str]:
input_dir = Path(input_directory)
output_path = Path(output_file)
suffix = output_path.suffix.lower()
if not input_dir.is_dir():
return None, f"Error: {input_dir} is not a directory"
if suffix not in {".docx", ".pptx", ".xlsx"}:
return None, f"Error: {output_file} must be a .docx, .pptx, or .xlsx file"
if validate and original_file:
original_path = Path(original_file)
if original_path.exists():
success, output = _run_validation(
input_dir, original_path, suffix, infer_author_func
)
if output:
print(output)
if not success:
return None, f"Error: Validation failed for {input_dir}"
with tempfile.TemporaryDirectory() as temp_dir:
temp_content_dir = Path(temp_dir) / "content"
shutil.copytree(input_dir, temp_content_dir)
for pattern in ["*.xml", "*.rels"]:
for xml_file in temp_content_dir.rglob(pattern):
_condense_xml(xml_file)
output_path.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zf:
for f in temp_content_dir.rglob("*"):
if f.is_file():
zf.write(f, f.relative_to(temp_content_dir))
return None, f"Successfully packed {input_dir} to {output_file}"
def _run_validation(
unpacked_dir: Path,
original_file: Path,
suffix: str,
infer_author_func=None,
) -> tuple[bool, str | None]:
output_lines = []
validators = []
if suffix == ".docx":
author = "Claude"
if infer_author_func:
try:
author = infer_author_func(unpacked_dir, original_file)
except ValueError as e:
print(f"Warning: {e} Using default author 'Claude'.", file=sys.stderr)
validators = [
DOCXSchemaValidator(unpacked_dir, original_file),
RedliningValidator(unpacked_dir, original_file, author=author),
]
elif suffix == ".pptx":
validators = [PPTXSchemaValidator(unpacked_dir, original_file)]
if not validators:
return True, None
total_repairs = sum(v.repair() for v in validators)
if total_repairs:
output_lines.append(f"Auto-repaired {total_repairs} issue(s)")
success = all(v.validate() for v in validators)
if success:
output_lines.append("All validations PASSED!")
return success, "\n".join(output_lines) if output_lines else None
def _condense_xml(xml_file: Path) -> None:
try:
with open(xml_file, encoding="utf-8") as f:
dom = defusedxml.minidom.parse(f)
for element in dom.getElementsByTagName("*"):
if element.tagName.endswith(":t"):
continue
for child in list(element.childNodes):
if (
child.nodeType == child.TEXT_NODE
and child.nodeValue
and child.nodeValue.strip() == ""
) or child.nodeType == child.COMMENT_NODE:
element.removeChild(child)
xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
except Exception as e:
print(f"ERROR: Failed to parse {xml_file.name}: {e}", file=sys.stderr)
raise
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Pack a directory into a DOCX, PPTX, or XLSX file"
)
parser.add_argument("input_directory", help="Unpacked Office document directory")
parser.add_argument("output_file", help="Output Office file (.docx/.pptx/.xlsx)")
parser.add_argument(
"--original",
help="Original file for validation comparison",
)
parser.add_argument(
"--validate",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Run validation with auto-repair (default: true)",
)
args = parser.parse_args()
_, message = pack(
args.input_directory,
args.output_file,
original_file=args.original,
validate=args.validate,
)
print(message)
if "Error" in message:
sys.exit(1)
skills/xlsx/scripts/office/soffice.py
+17 -8
View File
@@ -4,20 +4,23 @@ sockets may be blocked (e.g., sandboxed VMs). Detects the restriction
at runtime and applies an LD_PRELOAD shim if needed. at runtime and applies an LD_PRELOAD shim if needed.
Usage: Usage:
from office.soffice import run_soffice, get_soffice_env from office.soffice import run_soffice
# Option 1 – run soffice directly
result = run_soffice(["--headless", "--convert-to", "pdf", "input.docx"]) result = run_soffice(["--headless", "--convert-to", "pdf", "input.docx"])
# Option 2 – get env dict for your own subprocess calls Call soffice through run_soffice, not through subprocess with get_soffice_env():
env = get_soffice_env() the env dict carries the shim but names no user profile, and a non-root sandbox
subprocess.run(["soffice", ...], env=env) cannot bootstrap the default one -- soffice aborts with "User installation could
not be completed" and converts nothing. get_soffice_env() stays public for the
callers that build their own argv (they must pass -env:UserInstallation too).
""" """
import contextlib
import os import os
import socket import socket
import subprocess import subprocess
import tempfile import tempfile
from collections.abc import Iterable
from pathlib import Path from pathlib import Path
@@ -32,9 +35,15 @@ def get_soffice_env() -> dict:
return env return env
def run_soffice(args: list[str], **kwargs) -> subprocess.CompletedProcess: def run_soffice(args: Iterable[str], **kwargs) -> subprocess.CompletedProcess:
env = get_soffice_env() args = list(args)
return subprocess.run(["soffice"] + args, env=env, **kwargs) with contextlib.ExitStack() as stack:
if not any(str(a).startswith("-env:UserInstallation") for a in args):
profile = stack.enter_context(
tempfile.TemporaryDirectory(prefix="lo_profile_", ignore_cleanup_errors=True)
)
args = [f"-env:UserInstallation={Path(profile).as_uri()}"] + args
return subprocess.run(["soffice"] + args, env=get_soffice_env(), **kwargs)
skills/xlsx/scripts/office/unpack.py
-132
View File
@@ -1,132 +0,0 @@
"""Unpack Office files (DOCX, PPTX, XLSX) for editing.
Extracts the ZIP archive, pretty-prints XML files, and optionally:
- Merges adjacent runs with identical formatting (DOCX only)
- Simplifies adjacent tracked changes from same author (DOCX only)
Usage:
python unpack.py <office_file> <output_dir> [options]
Examples:
python unpack.py document.docx unpacked/
python unpack.py presentation.pptx unpacked/
python unpack.py document.docx unpacked/ --merge-runs false
"""
import argparse
import sys
import zipfile
from pathlib import Path
import defusedxml.minidom
from helpers.merge_runs import merge_runs as do_merge_runs
from helpers.simplify_redlines import simplify_redlines as do_simplify_redlines
SMART_QUOTE_REPLACEMENTS = {
"\u201c": "&#x201C;",
"\u201d": "&#x201D;",
"\u2018": "&#x2018;",
"\u2019": "&#x2019;",
}
def unpack(
input_file: str,
output_directory: str,
merge_runs: bool = True,
simplify_redlines: bool = True,
) -> tuple[None, str]:
input_path = Path(input_file)
output_path = Path(output_directory)
suffix = input_path.suffix.lower()
if not input_path.exists():
return None, f"Error: {input_file} does not exist"
if suffix not in {".docx", ".pptx", ".xlsx"}:
return None, f"Error: {input_file} must be a .docx, .pptx, or .xlsx file"
try:
output_path.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(input_path, "r") as zf:
zf.extractall(output_path)
xml_files = list(output_path.rglob("*.xml")) + list(output_path.rglob("*.rels"))
for xml_file in xml_files:
_pretty_print_xml(xml_file)
message = f"Unpacked {input_file} ({len(xml_files)} XML files)"
if suffix == ".docx":
if simplify_redlines:
simplify_count, _ = do_simplify_redlines(str(output_path))
message += f", simplified {simplify_count} tracked changes"
if merge_runs:
merge_count, _ = do_merge_runs(str(output_path))
message += f", merged {merge_count} runs"
for xml_file in xml_files:
_escape_smart_quotes(xml_file)
return None, message
except zipfile.BadZipFile:
return None, f"Error: {input_file} is not a valid Office file"
except Exception as e:
return None, f"Error unpacking: {e}"
def _pretty_print_xml(xml_file: Path) -> None:
try:
content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content)
xml_file.write_bytes(dom.toprettyxml(indent=" ", encoding="utf-8"))
except Exception:
pass
def _escape_smart_quotes(xml_file: Path) -> None:
try:
content = xml_file.read_text(encoding="utf-8")
for char, entity in SMART_QUOTE_REPLACEMENTS.items():
content = content.replace(char, entity)
xml_file.write_text(content, encoding="utf-8")
except Exception:
pass
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Unpack an Office file (DOCX, PPTX, XLSX) for editing"
)
parser.add_argument("input_file", help="Office file to unpack")
parser.add_argument("output_directory", help="Output directory")
parser.add_argument(
"--merge-runs",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Merge adjacent runs with identical formatting (DOCX only, default: true)",
)
parser.add_argument(
"--simplify-redlines",
type=lambda x: x.lower() == "true",
default=True,
metavar="true|false",
help="Merge adjacent tracked changes from same author (DOCX only, default: true)",
)
args = parser.parse_args()
_, message = unpack(
args.input_file,
args.output_directory,
merge_runs=args.merge_runs,
simplify_redlines=args.simplify_redlines,
)
print(message)
if "Error" in message:
sys.exit(1)
skills/xlsx/scripts/office/validate.py
+89 -27
View File
@@ -6,7 +6,7 @@ Usage:
The first argument can be either: The first argument can be either:
- An unpacked directory containing the Office document XML files - An unpacked directory containing the Office document XML files
- A packed Office file (.docx/.pptx/.xlsx) which will be unpacked to a temp directory - A packed Office file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx template) which will be unpacked to a temp directory
Auto-repair fixes: Auto-repair fixes:
- paraId/durableId values that exceed OOXML limits - paraId/durableId values that exceed OOXML limits
@@ -19,20 +19,43 @@ import tempfile
import zipfile import zipfile
from pathlib import Path from pathlib import Path
import defusedxml.ElementTree as ET
from defusedxml.common import DefusedXmlException
from helpers import OOXML_FAMILY, rezip, safe_extract
from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator
WORD_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
def _fail(message: str):
print(f"Error: {message}", file=sys.stderr)
sys.exit(2)
def _has_tracked_changes(unpacked_dir: Path) -> bool:
document = unpacked_dir / "word" / "document.xml"
if not document.is_file():
return False
try:
root = ET.parse(document).getroot()
except (ET.ParseError, DefusedXmlException):
return False
tracked = {f"{{{WORD_NS}}}ins", f"{{{WORD_NS}}}del"}
return any(elem.tag in tracked for elem in root.iter())
def main(): def main():
parser = argparse.ArgumentParser(description="Validate Office document XML files") parser = argparse.ArgumentParser(description="Validate Office document XML files")
parser.add_argument( parser.add_argument(
"path", "path",
help="Path to unpacked directory or packed Office file (.docx/.pptx/.xlsx)", help="Path to unpacked directory or packed Office file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx)",
) )
parser.add_argument( parser.add_argument(
"--original", "--original",
required=False, required=False,
default=None, default=None,
help="Path to original file (.docx/.pptx/.xlsx). If omitted, all XSD errors are reported and redlining validation is skipped.", help="Path to original file (.docx/.pptx/.xlsx or .dotx/.potx/.xltx). If omitted, all XSD errors are reported and redlining validation is skipped.",
) )
parser.add_argument( parser.add_argument(
"-v", "-v",
@@ -43,63 +66,102 @@ def main():
parser.add_argument( parser.add_argument(
"--auto-repair", "--auto-repair",
action="store_true", action="store_true",
help="Automatically repair common issues (hex IDs, whitespace preservation)", help="Automatically repair common issues (hex IDs, whitespace preservation). "
"Modifies the input in place: repairs to a packed file are written back to it.",
) )
parser.add_argument( parser.add_argument(
"--author", "--author",
default="Claude", default=None,
help="Author name for redlining validation (default: Claude)", help="The name you are redlining under. Passing it turns on the "
"tracked-change check: any text differing from --original without a "
"<w:ins>/<w:del> recording it is reported. Untracked edits carry no "
"author, so the check covers them whoever made them — the name marks "
"the run as redlining work and is not used to filter. Requires "
"--original; docx only.",
) )
args = parser.parse_args() args = parser.parse_args()
if args.author is not None and not args.original:
_fail("--author requires --original")
path = Path(args.path) path = Path(args.path)
assert path.exists(), f"Error: {path} does not exist" if not path.exists():
_fail(f"{path} does not exist")
original_file = None original_file = None
if args.original: if args.original:
original_file = Path(args.original) original_file = Path(args.original)
assert original_file.is_file(), f"Error: {original_file} is not a file" if not original_file.is_file():
assert original_file.suffix.lower() in [".docx", ".pptx", ".xlsx"], ( _fail(f"{original_file} is not a file")
f"Error: {original_file} must be a .docx, .pptx, or .xlsx file" if original_file.suffix.lower() not in OOXML_FAMILY:
_fail(f"{original_file} must be one of: {', '.join(sorted(OOXML_FAMILY))}")
family = OOXML_FAMILY.get((original_file or path).suffix.lower())
if family is None:
_fail(
f"Cannot determine file type from {path}. Use --original or provide one of: {', '.join(sorted(OOXML_FAMILY))}."
) )
file_extension = (original_file or path).suffix.lower() if args.author is not None and family != "docx":
assert file_extension in [".docx", ".pptx", ".xlsx"], ( _fail(f"--author only applies to docx files, not {family}")
f"Error: Cannot determine file type from {path}. Use --original or provide a .docx/.pptx/.xlsx file."
)
if path.is_file() and path.suffix.lower() in [".docx", ".pptx", ".xlsx"]: packed_file = None
temp_dir = tempfile.mkdtemp() temp_dir_ctx = None
with zipfile.ZipFile(path, "r") as zf: if path.is_file() and path.suffix.lower() in OOXML_FAMILY:
zf.extractall(temp_dir) packed_file = path
unpacked_dir = Path(temp_dir) temp_dir_ctx = tempfile.TemporaryDirectory()
unpacked_dir = Path(temp_dir_ctx.name)
try:
with zipfile.ZipFile(path, "r") as zf:
safe_extract(zf, unpacked_dir)
except (zipfile.BadZipFile, ValueError, OSError) as e:
_fail(f"cannot unpack {path}: {e}")
else: else:
assert path.is_dir(), f"Error: {path} is not a directory or Office file" if not path.is_dir():
_fail(f"{path} is not a directory or Office file")
unpacked_dir = path unpacked_dir = path
match file_extension: match family:
case ".docx": case "docx":
validators = [ validators = [
DOCXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), DOCXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose),
] ]
if original_file: if args.author is not None:
validators.append( validators.append(
RedliningValidator(unpacked_dir, original_file, verbose=args.verbose, author=args.author) RedliningValidator(unpacked_dir, original_file, verbose=args.verbose)
) )
case ".pptx": elif original_file and _has_tracked_changes(unpacked_dir):
print(
"Note: this document has tracked changes; they were not "
"checked against the original (pass --author to check)."
)
case "pptx":
validators = [ validators = [
PPTXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), PPTXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose),
] ]
case "xlsx":
exts = ", ".join(k for k, v in sorted(OOXML_FAMILY.items()) if v == "xlsx")
print(
f"No XSD schema validation is performed for xlsx-family files ({exts}). "
"For formula-error checking, use scripts/recalc.py instead."
)
sys.exit(0)
case _: case _:
print(f"Error: Validation not supported for file type {file_extension}") print(f"Error: Validation not supported for file type {family}")
sys.exit(1) sys.exit(1)
if args.auto_repair: if args.auto_repair:
total_repairs = sum(v.repair() for v in validators) total_repairs = sum(v.repair() for v in validators)
if total_repairs: if total_repairs:
print(f"Auto-repaired {total_repairs} issue(s)") print(f"Auto-repaired {total_repairs} issue(s)")
if packed_file is not None:
rezip(unpacked_dir, packed_file)
print(f"Wrote repaired file to {packed_file}")
success = all(v.validate() for v in validators) success = all([v.validate() for v in validators])
if temp_dir_ctx is not None:
temp_dir_ctx.cleanup()
if success: if success:
print("All validations PASSED!") print("All validations PASSED!")
skills/xlsx/scripts/office/validators/base.py
+48 -20
View File
@@ -6,8 +6,20 @@ import re
from pathlib import Path from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
from functools import lru_cache
import lxml.etree import lxml.etree
from helpers import safe_extract
@lru_cache(maxsize=None)
def _load_schema(schema_path: str):
with open(schema_path, "rb") as xsd_file:
xsd_doc = lxml.etree.parse(
xsd_file, parser=lxml.etree.XMLParser(), base_url=schema_path
)
return lxml.etree.XMLSchema(xsd_doc)
class BaseSchemaValidator: class BaseSchemaValidator:
@@ -119,21 +131,28 @@ class BaseSchemaValidator:
try: try:
content = xml_file.read_text(encoding="utf-8") content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content) dom = defusedxml.minidom.parseString(content)
modified = False pending = []
for elem in dom.getElementsByTagName("*"): for elem in dom.getElementsByTagName("*"):
if elem.tagName.endswith(":t") and elem.firstChild: local_name = elem.tagName.rsplit(":", 1)[-1]
text = elem.firstChild.nodeValue if local_name in ("t", "delText", "instrText", "delInstrText"):
if text and (text.startswith((' ', '\t')) or text.endswith((' ', '\t'))): text = "".join(
child.data
for child in elem.childNodes
if child.nodeType in (child.TEXT_NODE, child.CDATA_SECTION_NODE)
)
ws = (" ", "\t", "\n", "\r")
if text and (text.startswith(ws) or text.endswith(ws)):
if elem.getAttribute("xml:space") != "preserve": if elem.getAttribute("xml:space") != "preserve":
elem.setAttribute("xml:space", "preserve") elem.setAttribute("xml:space", "preserve")
text_preview = repr(text[:30]) + "..." if len(text) > 30 else repr(text) text_preview = repr(text[:30]) + "..." if len(text) > 30 else repr(text)
print(f" Repaired: {xml_file.name}: Added xml:space='preserve' to {elem.tagName}: {text_preview}") pending.append(f" Repaired: {xml_file.name}: Added xml:space='preserve' to {elem.tagName}: {text_preview}")
repairs += 1
modified = True
if modified: if pending:
xml_file.write_bytes(dom.toxml(encoding="UTF-8")) xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
for message in pending:
print(message)
repairs += len(pending)
except Exception: except Exception:
pass pass
@@ -212,6 +231,8 @@ class BaseSchemaValidator:
elem.getparent().remove(elem) elem.getparent().remove(elem)
for elem in root.iter(): for elem in root.iter():
if not hasattr(elem, "tag") or callable(elem.tag):
continue
tag = ( tag = (
elem.tag.split("}")[-1].lower() elem.tag.split("}")[-1].lower()
if "}" in elem.tag if "}" in elem.tag
@@ -326,6 +347,8 @@ class BaseSchemaValidator:
namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE}, namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE},
): ):
target = rel.get("Target") target = rel.get("Target")
if rel.get("TargetMode") == "External":
continue
if target and not target.startswith( if target and not target.startswith(
("http", "mailto:") ("http", "mailto:")
): ):
@@ -423,6 +446,8 @@ class BaseSchemaValidator:
r_ns = self.OFFICE_RELATIONSHIPS_NAMESPACE r_ns = self.OFFICE_RELATIONSHIPS_NAMESPACE
rid_attrs_to_check = ["id", "embed", "link"] rid_attrs_to_check = ["id", "embed", "link"]
for elem in xml_root.iter(): for elem in xml_root.iter():
if not hasattr(elem, "tag") or callable(elem.tag):
continue
for attr_name in rid_attrs_to_check: for attr_name in rid_attrs_to_check:
rid_attr = elem.get(f"{{{r_ns}}}{attr_name}") rid_attr = elem.get(f"{{{r_ns}}}{attr_name}")
if not rid_attr: if not rid_attr:
@@ -747,18 +772,16 @@ class BaseSchemaValidator:
return xml_doc return xml_doc
def _validate_single_file_xsd(self, xml_file, base_path): def _preprocess_for_schema(self, xml_doc, relative_path):
schema_path = self._get_schema_path(xml_file) return xml_doc
def _validate_single_file_xsd(self, xml_file, base_path, schema_path=None):
schema_path = schema_path or self._get_schema_path(xml_file)
if not schema_path: if not schema_path:
return None, None return None, None
try: try:
with open(schema_path, "rb") as xsd_file: schema = _load_schema(str(schema_path))
parser = lxml.etree.XMLParser()
xsd_doc = lxml.etree.parse(
xsd_file, parser=parser, base_url=str(schema_path)
)
schema = lxml.etree.XMLSchema(xsd_doc)
with open(xml_file, "r") as f: with open(xml_file, "r") as f:
xml_doc = lxml.etree.parse(f) xml_doc = lxml.etree.parse(f)
@@ -773,6 +796,8 @@ class BaseSchemaValidator:
): ):
xml_doc = self._clean_ignorable_namespaces(xml_doc) xml_doc = self._clean_ignorable_namespaces(xml_doc)
xml_doc = self._preprocess_for_schema(xml_doc, relative_path)
if schema.validate(xml_doc): if schema.validate(xml_doc):
return True, set() return True, set()
else: else:
@@ -784,7 +809,7 @@ class BaseSchemaValidator:
except Exception as e: except Exception as e:
return False, {str(e)} return False, {str(e)}
def _get_original_file_errors(self, xml_file): def _get_original_file_errors(self, xml_file, schema_path=None):
if self.original_file is None: if self.original_file is None:
return set() return set()
@@ -798,8 +823,11 @@ class BaseSchemaValidator:
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir) temp_path = Path(temp_dir)
with zipfile.ZipFile(self.original_file, "r") as zip_ref: try:
zip_ref.extractall(temp_path) with zipfile.ZipFile(self.original_file, "r") as zip_ref:
safe_extract(zip_ref, temp_path)
except (zipfile.BadZipFile, ValueError, OSError):
return set()
original_xml_file = temp_path / relative_path original_xml_file = temp_path / relative_path
@@ -807,7 +835,7 @@ class BaseSchemaValidator:
return set() return set()
is_valid, errors = self._validate_single_file_xsd( is_valid, errors = self._validate_single_file_xsd(
original_xml_file, temp_path original_xml_file, temp_path, schema_path=schema_path
) )
return errors if errors else set() return errors if errors else set()
skills/xlsx/scripts/office/validators/docx.py
+56 -36
View File
@@ -6,10 +6,13 @@ import random
import re import re
import tempfile import tempfile
import zipfile import zipfile
from pathlib import Path
import defusedxml.minidom import defusedxml.minidom
import lxml.etree import lxml.etree
from helpers import safe_extract
from .base import BaseSchemaValidator from .base import BaseSchemaValidator
@@ -186,7 +189,7 @@ class DOCXSchemaValidator(BaseSchemaValidator):
try: try:
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
with zipfile.ZipFile(original, "r") as zip_ref: with zipfile.ZipFile(original, "r") as zip_ref:
zip_ref.extractall(temp_dir) safe_extract(zip_ref, Path(temp_dir))
doc_xml_path = temp_dir + "/word/document.xml" doc_xml_path = temp_dir + "/word/document.xml"
root = lxml.etree.parse(doc_xml_path).getroot() root = lxml.etree.parse(doc_xml_path).getroot()
@@ -241,9 +244,12 @@ class DOCXSchemaValidator(BaseSchemaValidator):
return True return True
def compare_paragraph_counts(self): def compare_paragraph_counts(self):
original_count = self.count_paragraphs_in_original()
new_count = self.count_paragraphs_in_unpacked() new_count = self.count_paragraphs_in_unpacked()
if self.original_file is None:
print(f"\nParagraphs: {new_count}")
return
original_count = self.count_paragraphs_in_original()
diff = new_count - original_count diff = new_count - original_count
diff_str = f"+{diff}" if diff > 0 else str(diff) diff_str = f"+{diff}" if diff > 0 else str(diff)
print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})") print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})")
@@ -260,9 +266,15 @@ class DOCXSchemaValidator(BaseSchemaValidator):
try: try:
for elem in lxml.etree.parse(str(xml_file)).iter(): for elem in lxml.etree.parse(str(xml_file)).iter():
if val := elem.get(para_id_attr): if val := elem.get(para_id_attr):
if self._parse_id_value(val, base=16) >= 0x80000000: try:
if self._parse_id_value(val, base=16) >= 0x80000000:
errors.append(
f" {xml_file.name}:{elem.sourceline}: paraId={val} >= 0x80000000"
)
except ValueError:
errors.append( errors.append(
f" {xml_file.name}:{elem.sourceline}: paraId={val} >= 0x80000000" f" {xml_file.name}:{elem.sourceline}: "
f"paraId={val} is not valid hex"
) )
if val := elem.get(durable_id_attr): if val := elem.get(durable_id_attr):
@@ -279,13 +291,19 @@ class DOCXSchemaValidator(BaseSchemaValidator):
f"durableId={val} must be decimal in numbering.xml" f"durableId={val} must be decimal in numbering.xml"
) )
else: else:
if self._parse_id_value(val, base=16) >= 0x7FFFFFFF: try:
if self._parse_id_value(val, base=16) >= 0x7FFFFFFF:
errors.append(
f" {xml_file.name}:{elem.sourceline}: "
f"durableId={val} >= 0x7FFFFFFF"
)
except ValueError:
errors.append( errors.append(
f" {xml_file.name}:{elem.sourceline}: " f" {xml_file.name}:{elem.sourceline}: "
f"durableId={val} >= 0x7FFFFFFF" f"durableId={val} is not valid hex"
) )
except Exception: except lxml.etree.XMLSyntaxError:
pass continue
if errors: if errors:
print(f"FAILED - {len(errors)} ID constraint violations:") print(f"FAILED - {len(errors)} ID constraint violations:")
@@ -389,52 +407,54 @@ class DOCXSchemaValidator(BaseSchemaValidator):
return repairs return repairs
def repair_durableId(self) -> int: def repair_durableId(self) -> int:
DURABLE_ID_ATTRS = ("w16cid:durableId", "w16cex:durableId")
repairs = 0 repairs = 0
renames: dict = {}
for xml_file in self.xml_files: for xml_file in self.xml_files:
try: try:
content = xml_file.read_text(encoding="utf-8") content = xml_file.read_text(encoding="utf-8")
dom = defusedxml.minidom.parseString(content) dom = defusedxml.minidom.parseString(content)
is_numbering = xml_file.name == "numbering.xml"
base = 10 if is_numbering else 16
pending = []
seen_in_file = set()
modified = False modified = False
for elem in dom.getElementsByTagName("*"): for elem in dom.getElementsByTagName("*"):
if not elem.hasAttribute("w16cid:durableId"): for attr_name in DURABLE_ID_ATTRS:
continue if not elem.hasAttribute(attr_name):
continue
durable_id = elem.getAttribute("w16cid:durableId") durable_id = elem.getAttribute(attr_name)
needs_repair = False
if xml_file.name == "numbering.xml":
try: try:
needs_repair = ( key = self._parse_id_value(durable_id, base=base)
self._parse_id_value(durable_id, base=10) >= 0x7FFFFFFF needs_repair = key >= 0x7FFFFFFF
)
except ValueError:
needs_repair = True
else:
try:
needs_repair = (
self._parse_id_value(durable_id, base=16) >= 0x7FFFFFFF
)
except ValueError: except ValueError:
key = durable_id
needs_repair = True needs_repair = True
if needs_repair: if needs_repair:
value = random.randint(1, 0x7FFFFFFE) if key in seen_in_file:
if xml_file.name == "numbering.xml": value = random.randint(1, 0x7FFFFFFE)
new_id = str(value) else:
else: seen_in_file.add(key)
new_id = f"{value:08X}" if key not in renames:
renames[key] = random.randint(1, 0x7FFFFFFE)
value = renames[key]
new_id = str(value) if is_numbering else f"{value:08X}"
elem.setAttribute("w16cid:durableId", new_id) elem.setAttribute(attr_name, new_id)
print( pending.append(
f" Repaired: {xml_file.name}: durableId {durable_id} → {new_id}" f" Repaired: {xml_file.name}: durableId {durable_id} → {new_id}"
) )
repairs += 1 modified = True
modified = True
if modified: if modified:
xml_file.write_bytes(dom.toxml(encoding="UTF-8")) xml_file.write_bytes(dom.toxml(encoding="UTF-8"))
for message in pending:
print(message)
repairs += len(pending)
except Exception: except Exception:
pass pass
skills/xlsx/scripts/office/validators/pptx.py
+173 -7
View File
@@ -3,6 +3,9 @@ Validator for PowerPoint presentation XML files against XSD schemas.
""" """
import re import re
from pathlib import Path
from helpers import opc_target, rels_source_part, safe_extract
from .base import BaseSchemaValidator from .base import BaseSchemaValidator
@@ -57,8 +60,171 @@ class PPTXSchemaValidator(BaseSchemaValidator):
if not self.validate_no_duplicate_slide_layouts(): if not self.validate_no_duplicate_slide_layouts():
all_valid = False all_valid = False
if not self.validate_master_theme_uniqueness():
all_valid = False
if not self.validate_charts():
all_valid = False
if not self.validate_slides():
all_valid = False
return all_valid return all_valid
def _package_map(self) -> dict:
wanted = []
wanted += list(self.unpacked_dir.glob("[[]Content_Types[]].xml"))
wanted += list(self.unpacked_dir.glob("ppt/presentation.xml"))
wanted += list(self.unpacked_dir.glob("ppt/theme/*.xml"))
wanted += list(self.unpacked_dir.glob("ppt/theme/_rels/*.rels"))
wanted += list(self.unpacked_dir.glob("ppt/charts/chart*.xml"))
for group in ("slideMasters", "notesMasters", "handoutMasters"):
wanted += list(self.unpacked_dir.glob(f"ppt/{group}/*.xml"))
wanted += list(self.unpacked_dir.glob(f"ppt/{group}/_rels/*.rels"))
return {
p.relative_to(self.unpacked_dir).as_posix(): p.read_bytes()
for p in wanted
if p.is_file()
}
def validate_master_theme_uniqueness(self):
from helpers.pptx_theme import _NOTES_MASTERS, live_shared_master_themes
shared = live_shared_master_themes(self._package_map())
if shared:
print(f"FAILED - Found {len(shared)} master(s) sharing a theme part:")
for message in shared:
print(f" {message}")
if any(m.startswith(_NOTES_MASTERS) for m in shared):
print(" Fix: in ppt/presentation.xml, move <p:notesMasterIdLst> back to "
"directly after <p:sldIdLst>. PowerPoint reads that happily.")
else:
print(" Fix: give each master its own theme part.")
return False
if self.verbose:
print("PASSED - No master shares a theme part in a way PowerPoint refuses")
return True
def validate_charts(self):
from helpers.pptx_chart import find_chart_problems
problems = find_chart_problems(self._package_map())
if problems:
print(f"FAILED - Found {len(problems)} chart problem(s) PowerPoint rejects:")
for message in problems:
print(f" {message}")
return False
if self.verbose:
print("PASSED - Charts satisfy the constraints PowerPoint enforces")
return True
def _original_slide_defects(self, schema) -> set[str]:
import tempfile
import zipfile
from helpers.pptx_slide import SLIDE_PART_RE, fatal_slide_errors
if self.original_file is None:
return set()
found: set[str] = set()
with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir)
try:
with zipfile.ZipFile(self.original_file, "r") as zf:
safe_extract(zf, temp_path)
except (zipfile.BadZipFile, ValueError, OSError):
return set()
for part in sorted(temp_path.rglob("*.xml")):
relative = part.relative_to(temp_path).as_posix()
if not SLIDE_PART_RE.fullmatch(relative):
continue
ok, errors = self._validate_single_file_xsd(
part.resolve(), temp_path.resolve(), schema_path=schema
)
if ok is None or ok or not errors:
continue
found |= set(fatal_slide_errors(set(errors)))
return found
def validate_slides(self):
from helpers.pptx_slide import (
SLIDE_PART_RE,
fatal_slide_errors,
is_schema_verdict,
)
schema = self.schemas_dir / self.SCHEMA_MAPPINGS["ppt"]
inherited = self._original_slide_defects(schema)
problems: list[str] = []
broken: list[str] = []
for xml_file in self.xml_files:
relative = xml_file.relative_to(self.unpacked_dir).as_posix()
if not SLIDE_PART_RE.fullmatch(relative):
continue
ok, errors = self._validate_single_file_xsd(
xml_file.resolve(), self.unpacked_dir.resolve(), schema_path=schema
)
if ok is None or not errors:
continue
unreadable = [f"{relative}: {e}" for e in errors if not is_schema_verdict(e)]
if unreadable:
broken.extend(unreadable)
continue
if ok:
continue
for message in fatal_slide_errors(set(errors)):
if message in inherited:
continue
problems.append(f"{relative}: {message}")
if broken:
print(f"FAILED - Could not check {len(broken)} slide part(s):")
for message in sorted(broken):
print(f" {message[:240]}")
if problems:
print(f"FAILED - Found {len(problems)} slide problem(s) PowerPoint rejects:")
for message in sorted(problems):
print(f" {message[:240]}")
if broken or problems:
return False
if self.verbose:
print("PASSED - Slide XML has none of the defects PowerPoint refuses")
return True
def _get_schema_path(self, xml_file):
if xml_file.parent.name == "charts" and xml_file.name.startswith("chart"):
return None
return super()._get_schema_path(xml_file)
def _preprocess_for_schema(self, xml_doc, relative_path):
if relative_path.as_posix() != "ppt/presentation.xml":
return xml_doc
root = xml_doc.getroot()
ns = f"{{{self.PRESENTATIONML_NAMESPACE}}}"
notes = root.find(f"{ns}notesMasterIdLst")
slides = root.find(f"{ns}sldIdLst")
if notes is None or slides is None:
return xml_doc
children = list(root)
if children.index(notes) < children.index(slides):
return xml_doc
root.remove(notes)
root.insert(list(root).index(slides), notes)
return xml_doc
def validate_uuid_ids(self): def validate_uuid_ids(self):
import lxml.etree import lxml.etree
@@ -229,17 +395,17 @@ class PPTXSchemaValidator(BaseSchemaValidator):
): ):
rel_type = rel.get("Type", "") rel_type = rel.get("Type", "")
if "notesSlide" in rel_type: if "notesSlide" in rel_type:
target = rel.get("Target", "") part = opc_target(
if target: rel.get("Target", ""),
normalized_target = target.replace("../", "") rels_source_part(rels_file, self.unpacked_dir),
rel.get("TargetMode", ""),
)
if part:
slide_name = rels_file.stem.replace( slide_name = rels_file.stem.replace(
".xml", "" ".xml", ""
) )
if normalized_target not in notes_slide_references: notes_slide_references.setdefault(part, []).append(
notes_slide_references[normalized_target] = []
notes_slide_references[normalized_target].append(
(slide_name, rels_file) (slide_name, rels_file)
) )
skills/xlsx/scripts/office/validators/redlining.py
+96 -44
View File
@@ -1,5 +1,14 @@
""" """
Validator for tracked changes in Word documents. Validator for tracked changes in Word documents.
Detects untracked edits in word/document.xml: text that differs from the
original without a <w:ins>/<w:del> wrapper recording it. The tracked changes
that are new relative to the original are undone, and the result is compared
against the original; whatever text still differs was edited without being
tracked.
Only the document body is compared. Headers, footers, footnotes and endnotes
are separate parts and are not checked.
""" """
import subprocess import subprocess
@@ -7,14 +16,18 @@ import tempfile
import zipfile import zipfile
from pathlib import Path from pathlib import Path
import defusedxml.ElementTree as ET
from defusedxml.common import DefusedXmlException
from helpers import rendered_text, safe_extract
class RedliningValidator: class RedliningValidator:
def __init__(self, unpacked_dir, original_docx, verbose=False, author="Claude"): def __init__(self, unpacked_dir, original_docx, verbose=False):
self.unpacked_dir = Path(unpacked_dir) self.unpacked_dir = Path(unpacked_dir)
self.original_docx = Path(original_docx) self.original_docx = Path(original_docx)
self.verbose = verbose self.verbose = verbose
self.author = author
self.namespaces = { self.namespaces = {
"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main" "w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
} }
@@ -28,40 +41,12 @@ class RedliningValidator:
print(f"FAILED - Modified document.xml not found at {modified_file}") print(f"FAILED - Modified document.xml not found at {modified_file}")
return False return False
try:
import xml.etree.ElementTree as ET
tree = ET.parse(modified_file)
root = tree.getroot()
del_elements = root.findall(".//w:del", self.namespaces)
ins_elements = root.findall(".//w:ins", self.namespaces)
author_del_elements = [
elem
for elem in del_elements
if elem.get(f"{{{self.namespaces['w']}}}author") == self.author
]
author_ins_elements = [
elem
for elem in ins_elements
if elem.get(f"{{{self.namespaces['w']}}}author") == self.author
]
if not author_del_elements and not author_ins_elements:
if self.verbose:
print(f"PASSED - No tracked changes by {self.author} found.")
return True
except Exception:
pass
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir) temp_path = Path(temp_dir)
try: try:
with zipfile.ZipFile(self.original_docx, "r") as zip_ref: with zipfile.ZipFile(self.original_docx, "r") as zip_ref:
zip_ref.extractall(temp_path) safe_extract(zip_ref, temp_path)
except Exception as e: except Exception as e:
print(f"FAILED - Error unpacking original docx: {e}") print(f"FAILED - Error unpacking original docx: {e}")
return False return False
@@ -74,18 +59,16 @@ class RedliningValidator:
return False return False
try: try:
import xml.etree.ElementTree as ET
modified_tree = ET.parse(modified_file) modified_tree = ET.parse(modified_file)
modified_root = modified_tree.getroot() modified_root = modified_tree.getroot()
original_tree = ET.parse(original_file) original_tree = ET.parse(original_file)
original_root = original_tree.getroot() original_root = original_tree.getroot()
except ET.ParseError as e: except (ET.ParseError, DefusedXmlException) as e:
print(f"FAILED - Error parsing XML files: {e}") print(f"FAILED - Error parsing XML files: {e}")
return False return False
self._remove_author_tracked_changes(original_root) new_changes = self._new_tracked_changes(original_root, modified_root)
self._remove_author_tracked_changes(modified_root) self._remove_tracked_changes(modified_root, new_changes)
modified_text = self._extract_text_content(modified_root) modified_text = self._extract_text_content(modified_root)
original_text = self._extract_text_content(original_root) original_text = self._extract_text_content(original_root)
@@ -98,20 +81,91 @@ class RedliningValidator:
return False return False
if self.verbose: if self.verbose:
print(f"PASSED - All changes by {self.author} are properly tracked") print(
f"PASSED - All {len(new_changes)} change(s) against the original "
"are properly tracked"
)
return True return True
def _tracked_change_elements(self, root):
ins_tag = f"{{{self.namespaces['w']}}}ins"
del_tag = f"{{{self.namespaces['w']}}}del"
return [elem for elem in root.iter() if elem.tag in (ins_tag, del_tag)]
def _rendered_text(self, elem):
preserve = elem.get("{http://www.w3.org/XML/1998/namespace}space") == "preserve"
return rendered_text(elem.text or "", preserve)
def _text_elements(self, elem):
w = self.namespaces["w"]
return [
node
for node in elem.iter()
if node.tag in (f"{{{w}}}t", f"{{{w}}}delText")
]
def _tracked_change_key(self, elem):
w = self.namespaces["w"]
text = "".join(self._rendered_text(node) for node in self._text_elements(elem))
return (elem.tag, elem.get(f"{{{w}}}author"), elem.get(f"{{{w}}}date"), text)
def _new_tracked_changes(self, original_root, modified_root):
original = self._tracked_change_elements(original_root)
modified = self._tracked_change_elements(modified_root)
pool = {}
for elem in original:
pool.setdefault(self._tracked_change_key(elem), []).append(elem)
matched, leftover = set(), []
for elem in modified:
bucket = pool.get(self._tracked_change_key(elem))
if bucket:
matched.add(bucket.pop())
else:
leftover.append(elem)
def group(elem):
return self._tracked_change_key(elem)[:3]
def text_of(elems):
return "".join(self._tracked_change_key(e)[3] for e in elems)
unmatched_original = {}
for elem in original:
if elem not in matched:
unmatched_original.setdefault(group(elem), []).append(elem)
by_group = {}
for elem in leftover:
by_group.setdefault(group(elem), []).append(elem)
new = set()
for key, elems in by_group.items():
rebuilt = text_of(elems)
if rebuilt and rebuilt == text_of(unmatched_original.get(key, [])):
continue
new.update(elems)
return new
def _generate_detailed_diff(self, original_text, modified_text): def _generate_detailed_diff(self, original_text, modified_text):
error_parts = [ error_parts = [
f"FAILED - Document text doesn't match after removing {self.author}'s tracked changes", "FAILED - Document text doesn't match after removing the tracked changes",
"", "",
"Likely causes:", "Likely causes:",
" 1. Modified text inside another author's <w:ins> or <w:del> tags", " 1. Modified text inside another author's <w:ins> or <w:del> tags",
" 2. Made edits without proper tracked changes", " 2. Made edits without proper tracked changes",
" 3. Didn't nest <w:del> inside <w:ins> when deleting another's insertion", " 3. Didn't nest <w:del> inside <w:ins> when deleting another's insertion",
" 4. Rewrote another author's <w:ins>/<w:del> and changed its text on",
" the way. A tracked change from the original is recognised by its",
" author, date and text; anything that doesn't reproduce one exactly",
" reads as new, and the text it carried is reported missing.",
"", "",
"For pre-redlined documents, use correct patterns:", "For pre-redlined documents, use correct patterns:",
" - To reject another's INSERTION: Nest <w:del> inside their <w:ins>", " - To reject another's INSERTION: Nest <w:del> inside their <w:ins>",
" - To reject PART of one: nest <w:del> around only the runs you reject.",
" Their <w:ins> may be split around it, so long as the pieces keep",
" their author and date and still spell out the same text.",
" - To restore another's DELETION: Add new <w:ins> AFTER their <w:del>", " - To restore another's DELETION: Add new <w:ins> AFTER their <w:del>",
"", "",
] ]
@@ -195,15 +249,14 @@ class RedliningValidator:
return None return None
def _remove_author_tracked_changes(self, root): def _remove_tracked_changes(self, root, targets):
ins_tag = f"{{{self.namespaces['w']}}}ins" ins_tag = f"{{{self.namespaces['w']}}}ins"
del_tag = f"{{{self.namespaces['w']}}}del" del_tag = f"{{{self.namespaces['w']}}}del"
author_attr = f"{{{self.namespaces['w']}}}author"
for parent in root.iter(): for parent in root.iter():
to_remove = [] to_remove = []
for child in parent: for child in parent:
if child.tag == ins_tag and child.get(author_attr) == self.author: if child.tag == ins_tag and child in targets:
to_remove.append(child) to_remove.append(child)
for elem in to_remove: for elem in to_remove:
parent.remove(elem) parent.remove(elem)
@@ -214,7 +267,7 @@ class RedliningValidator:
for parent in root.iter(): for parent in root.iter():
to_process = [] to_process = []
for child in parent: for child in parent:
if child.tag == del_tag and child.get(author_attr) == self.author: if child.tag == del_tag and child in targets:
to_process.append((child, list(parent).index(child))) to_process.append((child, list(parent).index(child)))
for del_elem, del_index in reversed(to_process): for del_elem, del_index in reversed(to_process):
@@ -234,8 +287,7 @@ class RedliningValidator:
for p_elem in root.findall(f".//{p_tag}"): for p_elem in root.findall(f".//{p_tag}"):
text_parts = [] text_parts = []
for t_elem in p_elem.findall(f".//{t_tag}"): for t_elem in p_elem.findall(f".//{t_tag}"):
if t_elem.text: text_parts.append(self._rendered_text(t_elem))
text_parts.append(t_elem.text)
paragraph_text = "".join(text_parts) paragraph_text = "".join(text_parts)
if paragraph_text: if paragraph_text:
paragraphs.append(paragraph_text) paragraphs.append(paragraph_text)
skills/xlsx/scripts/recalc.py
+168 -44
View File
@@ -3,20 +3,29 @@ Excel Formula Recalculation Script
Recalculates all formulas in an Excel file using LibreOffice Recalculates all formulas in an Excel file using LibreOffice
""" """
import contextlib
import json import json
import os import os
import platform import platform
import re
import shutil
import subprocess import subprocess
import sys import sys
import tempfile
import time
import zipfile
from pathlib import Path from pathlib import Path
from office.soffice import get_soffice_env from office.soffice import get_soffice_env, run_soffice
from openpyxl import load_workbook from openpyxl import load_workbook
MACRO_DIR_MACOS = "~/Library/Application Support/LibreOffice/4/user/basic/Standard"
MACRO_DIR_LINUX = "~/.config/libreoffice/4/user/basic/Standard"
MACRO_FILENAME = "Module1.xba" MACRO_FILENAME = "Module1.xba"
SOFFICE_MISSING = "soffice not found on PATH; LibreOffice is required to recalculate"
MAX_LOCATIONS = 100
EXTERNAL_REF_RE = re.compile(r"""(?<![\w"\[])'?\[\d+\][^!"\[\]]*'?!""")
RECALCULATE_MACRO = """<?xml version="1.0" encoding="UTF-8"?> RECALCULATE_MACRO = """<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE script:module PUBLIC "-//OpenOffice.org//DTD OfficeDocument 1.0//EN" "module.dtd"> <!DOCTYPE script:module PUBLIC "-//OpenOffice.org//DTD OfficeDocument 1.0//EN" "module.dtd">
@@ -39,63 +48,168 @@ def has_gtimeout():
return False return False
def setup_libreoffice_macro(): def _stamp(path):
macro_dir = os.path.expanduser( st = os.stat(path)
MACRO_DIR_MACOS if platform.system() == "Darwin" else MACRO_DIR_LINUX return st.st_mtime_ns, st.st_size
)
macro_file = os.path.join(macro_dir, MACRO_FILENAME)
if (
os.path.exists(macro_file)
and "RecalculateAndSave" in Path(macro_file).read_text()
):
return True
if not os.path.exists(macro_dir): def setup_libreoffice_macro(profile_dir: Path, timeout=30):
subprocess.run( url = profile_dir.as_uri()
["soffice", "--headless", "--terminate_after_init"], try:
run_soffice(
["--headless", "--terminate_after_init", f"-env:UserInstallation={url}"],
capture_output=True, capture_output=True,
timeout=10, timeout=timeout,
env=get_soffice_env(),
) )
os.makedirs(macro_dir, exist_ok=True) except FileNotFoundError:
return None, SOFFICE_MISSING
except subprocess.TimeoutExpired:
return None, "LibreOffice timed out creating its profile; formulas were NOT recalculated"
macro_dir = profile_dir / "user" / "basic" / "Standard"
if not macro_dir.exists():
return None, "LibreOffice did not create a usable profile; formulas were NOT recalculated"
try: try:
Path(macro_file).write_text(RECALCULATE_MACRO) (macro_dir / MACRO_FILENAME).write_text(RECALCULATE_MACRO)
return True except OSError as e:
except Exception: return None, f"Could not install the recalculation macro: {e}"
return False
return url, None
def recalc(filename, timeout=30): def external_links_at_risk(filename):
try:
with zipfile.ZipFile(filename) as archive:
names = archive.namelist()
except (zipfile.BadZipFile, OSError):
return []
if not any(n.startswith("xl/externalLinks/") for n in names):
return []
with contextlib.ExitStack() as stack:
formulas = load_workbook(filename, data_only=False)
stack.callback(formulas.close)
values = load_workbook(filename, data_only=True)
stack.callback(values.close)
external_names = [
name
for name, dn in formulas.defined_names.items()
if isinstance(getattr(dn, "value", None), str) and EXTERNAL_REF_RE.search(dn.value)
]
name_re = (
re.compile(r"\b(" + "|".join(re.escape(n) for n in external_names) + r")\b")
if external_names
else None
)
at_risk = []
for sheet in formulas.sheetnames:
ws = formulas[sheet]
if not hasattr(ws, "iter_rows"):
continue
cached = values[sheet]
for row in ws.iter_rows():
for cell in row:
v = cell.value
if not (isinstance(v, str) and v.startswith("=")):
continue
reaches_out = EXTERNAL_REF_RE.search(v) or (name_re and name_re.search(v))
if reaches_out and cached[cell.coordinate].value is None:
at_risk.append(f"{sheet}!{cell.coordinate}")
return at_risk
def recalc(filename, timeout=30, force=False):
if not Path(filename).exists(): if not Path(filename).exists():
return {"error": f"File {filename} does not exist"} return {"error": f"File {filename} does not exist"}
abs_path = str(Path(filename).absolute()) abs_path = str(Path(filename).absolute())
if not setup_libreoffice_macro(): if not os.access(abs_path, os.W_OK):
return {"error": "Failed to setup LibreOffice macro"} return {"error": f"{filename} is not writable; recalculation rewrites the file in place"}
try:
get_soffice_env()
except Exception as e:
return {"error": f"Could not prepare the LibreOffice environment: {e}"}
if not force:
try:
at_risk = external_links_at_risk(filename)
except Exception as e:
return {"error": f"Could not inspect {filename} for external links: {e}"}
if at_risk:
shown = at_risk[:MAX_LOCATIONS]
return {
"error": (
"Refusing to recalculate: this workbook links to another workbook, and "
f"{len(at_risk)} linked cell(s) have lost their cached value (openpyxl strips "
"these on save). Recalculating would resolve them to #NAME? and delete the "
"external links for good. Copy those cells' values from the original file "
"before saving, or pass --force to accept the loss. Charts and conditional "
"formats can hold external references too, so this list may not be exhaustive."
),
"external_link_cells": shown,
"external_link_cells_truncated": max(0, len(at_risk) - len(shown)),
}
with tempfile.TemporaryDirectory(
prefix="recalc-lo-profile-", ignore_cleanup_errors=True
) as profile_dir:
return _recalc_with_profile(filename, abs_path, timeout, Path(profile_dir))
def _recalc_with_profile(filename, abs_path, timeout, profile_dir: Path):
started = time.monotonic()
profile_url, err = setup_libreoffice_macro(profile_dir, timeout=timeout)
if err:
return {"error": err}
timeout = max(5, int(timeout - (time.monotonic() - started)))
before = _stamp(abs_path)
cmd = [ cmd = [
"soffice", "soffice",
"--headless", "--headless",
"--norestore", "--norestore",
f"-env:UserInstallation={profile_url}",
"vnd.sun.star.script:Standard.Module1.RecalculateAndSave?language=Basic&location=application", "vnd.sun.star.script:Standard.Module1.RecalculateAndSave?language=Basic&location=application",
abs_path, abs_path,
] ]
if platform.system() == "Linux": if platform.system() == "Linux" and shutil.which("timeout"):
cmd = ["timeout", str(timeout)] + cmd cmd = ["timeout", str(timeout)] + cmd
elif platform.system() == "Darwin" and has_gtimeout(): elif platform.system() == "Darwin" and has_gtimeout():
cmd = ["gtimeout", str(timeout)] + cmd cmd = ["gtimeout", str(timeout)] + cmd
result = subprocess.run(cmd, capture_output=True, text=True, env=get_soffice_env()) timed_out = f"LibreOffice timed out after {timeout}s; formulas were NOT recalculated. Re-run with a longer timeout."
if result.returncode != 0 and result.returncode != 124: try:
error_msg = result.stderr or "Unknown error during recalculation" result = subprocess.run(
if "Module1" in error_msg or "RecalculateAndSave" not in error_msg: cmd, capture_output=True, text=True, env=get_soffice_env(), timeout=timeout + 15
return {"error": "LibreOffice macro not configured properly"} )
return {"error": error_msg} except subprocess.TimeoutExpired:
return {"error": timed_out}
except FileNotFoundError:
return {"error": SOFFICE_MISSING}
if result.returncode == 124:
return {"error": timed_out}
if result.returncode != 0:
detail = (result.stderr or "").strip() or f"soffice exited {result.returncode}"
return {"error": f"LibreOffice failed to recalculate: {detail}"}
if _stamp(abs_path) == before:
return {
"error": (
"LibreOffice exited cleanly but never rewrote the file, so nothing was "
"recalculated. Check that no other LibreOffice instance is running, then retry."
)
}
try: try:
wb = load_workbook(filename, data_only=True) wb = load_workbook(filename, data_only=True)
@@ -114,6 +228,8 @@ def recalc(filename, timeout=30):
for sheet_name in wb.sheetnames: for sheet_name in wb.sheetnames:
ws = wb[sheet_name] ws = wb[sheet_name]
if not hasattr(ws, "iter_rows"):
continue
for row in ws.iter_rows(): for row in ws.iter_rows():
for cell in row: for cell in row:
if cell.value is not None and isinstance(cell.value, str): if cell.value is not None and isinstance(cell.value, str):
@@ -124,8 +240,6 @@ def recalc(filename, timeout=30):
total_errors += 1 total_errors += 1
break break
wb.close()
result = { result = {
"status": "success" if total_errors == 0 else "errors_found", "status": "success" if total_errors == 0 else "errors_found",
"total_errors": total_errors, "total_errors": total_errors,
@@ -134,15 +248,19 @@ def recalc(filename, timeout=30):
for err_type, locations in error_details.items(): for err_type, locations in error_details.items():
if locations: if locations:
result["error_summary"][err_type] = { entry = {"count": len(locations), "locations": locations[:MAX_LOCATIONS]}
"count": len(locations), if len(locations) > MAX_LOCATIONS:
"locations": locations[:20], entry["locations_truncated"] = len(locations) - MAX_LOCATIONS
} result["error_summary"][err_type] = entry
wb.close()
wb_formulas = load_workbook(filename, data_only=False) wb_formulas = load_workbook(filename, data_only=False)
formula_count = 0 formula_count = 0
for sheet_name in wb_formulas.sheetnames: for sheet_name in wb_formulas.sheetnames:
ws = wb_formulas[sheet_name] ws = wb_formulas[sheet_name]
if not hasattr(ws, "iter_rows"):
continue
for row in ws.iter_rows(): for row in ws.iter_rows():
for cell in row: for cell in row:
if ( if (
@@ -162,8 +280,11 @@ def recalc(filename, timeout=30):
def main(): def main():
if len(sys.argv) < 2: args = [a for a in sys.argv[1:] if a != "--force"]
print("Usage: python recalc.py <excel_file> [timeout_seconds]") force = "--force" in sys.argv[1:]
if not args:
print("Usage: python recalc.py <excel_file> [timeout_seconds] [--force]")
print("\nRecalculates all formulas in an Excel file using LibreOffice") print("\nRecalculates all formulas in an Excel file using LibreOffice")
print("\nReturns JSON with error details:") print("\nReturns JSON with error details:")
print(" - status: 'success' or 'errors_found'") print(" - status: 'success' or 'errors_found'")
@@ -171,13 +292,16 @@ def main():
print(" - total_formulas: Number of formulas in the file") print(" - total_formulas: Number of formulas in the file")
print(" - error_summary: Breakdown by error type with locations") print(" - error_summary: Breakdown by error type with locations")
print(" - #VALUE!, #DIV/0!, #REF!, #NAME?, #NULL!, #NUM!, #N/A") print(" - #VALUE!, #DIV/0!, #REF!, #NAME?, #NULL!, #NUM!, #N/A")
print("\nOn any failure the JSON has an 'error' key and no 'status'.")
print("--force recalculates even when it would destroy external links.")
sys.exit(1) sys.exit(1)
filename = sys.argv[1] filename = args[0]
timeout = int(sys.argv[2]) if len(sys.argv) > 2 else 30 timeout = int(args[1]) if len(args) > 1 else 30
result = recalc(filename, timeout) result = recalc(filename, timeout, force=force)
print(json.dumps(result, indent=2)) print(json.dumps(result, indent=2))
sys.exit(1 if "error" in result else 0)
if __name__ == "__main__": if __name__ == "__main__":