Skip to content

Latest commit

 

History

History
189 lines (147 loc) · 6.01 KB

MD_MAGIC.md

File metadata and controls

189 lines (147 loc) · 6.01 KB

Markdown Cell Magic Implementation Design

Problem Statement

Training participants need a way to copy Markdown cells from trainer notebooks during live sessions. Unlike code cells which have visible execution counts, Markdown cells lack visible identifiers, making them difficult to reference.

Design Considerations

Current Magic Commands

  • %code (formerly %nb): Handles code cells (identified by execution_count)
  • %md: Handles markdown cells with flexible syntax (by index or relative to code cells)
  • %nb: Kept as alias for %code for backward compatibility

Challenges with Markdown Cells

  1. No visible numbering: Markdown cells don't have execution counts
  2. Cell IDs not user-friendly: Internal IDs exist but aren't visible in the UI
  3. Mixed cell types: Notebooks interleave code and Markdown cells
  4. Cell type conversion: Content retrieved needs manual conversion from Code to Markdown

Proposed Solutions

Solution 1: Between Code Cells (Fence Method)

Syntax: %md 1 2 - Get all Markdown cells between code cells 1 and 2

Pros:

  • Uses existing visible code cell numbers
  • Intuitive for contiguous Markdown sections

Cons:

  • Ambiguous for non-contiguous selections
  • Doesn't work for Markdown at notebook start/end
  • May return multiple cells when only one is needed

Solution 2: Position-Based Reference

Syntax: %md after:3 or %md before:5 - Get Markdown cell(s) after/before code cell

Pros:

  • Precise single-cell selection
  • Clear intent
  • Works at notebook boundaries

Cons:

  • Requires multiple commands for multiple cells
  • User needs to know exact positions

Solution 3: Content Pattern Matching

Syntax: %md "# Section Title" - Find Markdown cells containing pattern

Pros:

  • Natural for users who can see content
  • Works without knowing cell positions
  • Can use distinctive headers/keywords

Cons:

  • Pattern might not be unique
  • Requires exact string matching or regex

Solution 4: Hybrid Cell Magic (Recommended)

Syntax: %%md 3 - Cell magic that fetches Markdown after code cell 3

Pros:

  • Automatically converts cell type to Markdown
  • No manual cleanup needed
  • Clear one-to-one mapping
  • Can extend to support ranges

Cons:

  • Different from line magic pattern
  • Requires cell magic implementation

Solution 5: Smart Sequential Indexing (Recommended Alternative)

Syntax: %md 3 or %md 3-5 - Use sequential index for all Markdown cells

Implementation:

  • Build index of all Markdown cells in order
  • Reference by position (1st, 2nd, 3rd Markdown cell)
  • Optional: %md --list to show available Markdown cells with previews

Pros:

  • Simple, consistent interface
  • Works like %nb for code cells
  • No ambiguity
  • Can show preview list

Cons:

  • Users need to count Markdown cells
  • Numbers change if trainer adds cells

Implemented Solution

The %md command now supports unified syntax for both index-based and code-relative selection:

Markdown Index Syntax

%md m1        # Get 1st Markdown cell
%md m3-m5     # Get 3rd through 5th Markdown cells
%md m7-       # Get 7th cell onwards
%md -m3       # Get cells 1 through 3
%md --list    # List all Markdown cells with m-prefixed numbers

Code-Relative Syntax (uses last occurrence if duplicates exist)

%md -3        # All Markdown cells before code cell 3
%md 5-        # All Markdown cells after code cell 5
%md 3-5       # All Markdown between code cells 3 and 5

Combined Usage

%md -1 m3 5-  # Before code 1, markdown cell 3, after code 5

Implementation Details

Cell Type Handling

  • Unlike %nb, the %md and %mdat magics do NOT add a comment header
  • Content is inserted directly without any prefix
  • Users manually convert cell type (Ctrl+M, M in Jupyter)
  • Future: Investigate IPython API for automatic cell type conversion

Preview Feature

%md --list
# Output:
# 1: # Introduction
#    This notebook covers...
# 2: ## Data Loading
#    We'll use pandas to...
# 3: ### Important Notes
#    Remember to check...

Error Handling

  • Clear messages for invalid ranges
  • Warnings when no Markdown cells found
  • Handle notebooks without code cells gracefully

Alternative Approaches Considered

Invisible Anchors

Add hidden HTML comments as anchors in Markdown cells:

<!-- md-anchor: section1 -->
# Introduction

Rejected: Requires modifying trainer notebooks

Visual Cell Numbers

Propose Jupyter enhancement to show Markdown cell numbers in UI. Rejected: Outside our control, long-term solution

Dual Output

Return both code and Markdown versions, let user choose. Rejected: Clutters interface, still requires manual work

Migration Path

  1. Renamed %nb to %code as the canonical command for code cells
  2. Kept %nb as an alias for backward compatibility
  3. Implemented unified %md command with both index and position-based syntax
  4. Removed separate %mdat command (functionality merged into %md)
  5. Updated all documentation and help text

Usage Examples

Trainer notebook structure:

[Code 1] Import statements
[Markdown] # Data Analysis Workshop
[Markdown] ## Prerequisites  
[Code 2] Load data
[Markdown] ### Understanding the dataset
[Code 3] Explore data

Participant commands:

%code 1        # Get code cell 1 (preferred)
%nb 1          # Same as %code 1 (backward compatibility)
%md m1         # Get "# Data Analysis Workshop"
%md m2-m3      # Get "## Prerequisites" and "### Understanding the dataset"
%md 1-         # Get markdown after code cell 1
%md -2         # Get markdown before code cell 2

Conclusion

The unified %md command provides a powerful and flexible interface for selecting markdown cells. By supporting both index-based (m1, m2-m5) and code-relative (-1, 2-, 3-5) syntax in a single command, users have maximum flexibility without needing to remember multiple commands. The implementation handles edge cases like duplicate code cell numbers (using the last occurrence) and provides clear error messages for invalid selections.