Skip to main content

v2.4.2+ - Complete Feature Export & Enhanced Progress

Release Date: October 12, 2025

Tagline: All Computed Features Now Exported - Complete ML-Ready Datasets

🎯 Overview​

Version 2.4.2+ addresses a critical feature export issue where only 12 features were being saved despite computing 35-45+ features in full mode. This release ensures all computed geometric features are exported to disk across all output formats (NPZ, HDF5, PyTorch, LAZ), with enhanced metadata tracking and improved progress reporting.

Dataset Regeneration Recommended

Datasets generated before v2.4.2+ may be missing critical features. We recommend regenerating training datasets to access the complete feature sets.

✨ What's New​

Complete Feature Export​

  • 📊 All Features Saved: Fixed critical bug where only 12/43 features were exported
  • 📝 Feature Metadata: Added metadata['feature_names'] list and metadata['num_features'] count
  • 🎯 Consistent Ordering: Features exported in reproducible order across all formats
  • 💾 Complete LAZ Export: All geometric features now saved as extra dimensions in LAZ files
  • 🔍 Feature Tracking: Easy verification of which features are in your datasets

Enhanced Progress Reporting​

  • 📈 Detailed Progress Bars: Point counts, chunk info, memory usage, and processing rates
  • 🎯 Mode Indicators: Visual indicators for GPU (🎯), GPU fallback (🔧), and CPU (💻) modes
  • ⏱️ Better Time Estimates: Accurate remaining time predictions for long-running jobs
  • 📊 Completion Statistics: Summary statistics after feature computation completes

📊 Feature Export Details​

Before v2.4.2 (Bug)​

Issue: Only 12 features exported despite computing 35-45+

# What was computed (43+ features):
# - Normals (3), Curvature (2), Eigenvalues (5), ...
# - Architectural (5), Density (5), Height (7), ...

# What was saved (only 12 features):
features = [
    'normal_x', 'normal_y', 'normal_z',
    'curvature', 'planarity', 'linearity',
    'sphericity', 'verticality', 'density',
    'height_above_ground', 'wall_score', 'roof_score'
]

After v2.4.2+ (Fixed)​

All Features Exported in consistent order:

# Normals (3)
['normal_x', 'normal_y', 'normal_z']

# Core shape descriptors (6)
['planarity', 'linearity', 'sphericity', 'anisotropy', 'roughness', 'omnivariance']

# Curvature features (2)
['curvature', 'change_curvature']

# Eigenvalue features (5)
['eigenvalue_1', 'eigenvalue_2', 'eigenvalue_3', 'sum_eigenvalues', 'eigenentropy']

# Height features (7)
['height_above_ground', 'vertical_std', 'z_normalized',
 'z_absolute', 'z_from_ground', 'z_from_median', 'distance_to_center']

# Building scores (3)
['verticality', 'wall_score', 'roof_score', 'horizontality']

# Density features (5)
['density', 'local_density', 'num_points_2m', 'neighborhood_extent', 'height_extent_ratio']

# Architectural features (5)
['edge_strength', 'corner_likelihood', 'overhang_indicator',
 'surface_roughness', 'local_roughness']

# Total: 38-43+ features depending on mode

🔧 Technical Changes​

Feature Matrix Construction​

Updated: base_formatter.py - _build_feature_matrix()

  • Returns all computed features in consistent order
  • Added return_feature_names=True parameter for metadata
  • Comprehensive feature ordering for reproducibility

LAZ Export Enhancement​

Updated: serialization.py - _add_geometric_features()

# Before: Only 8 features
geometric_features = [
    'planarity', 'linearity', 'sphericity', 'anisotropy',
    'roughness', 'density', 'curvature', 'verticality'
]

# After: All 35+ features
geometric_features = [
    # Core shape descriptors (6)
    'planarity', 'linearity', 'sphericity',
    'anisotropy', 'roughness', 'omnivariance',
    # Curvature (2)
    'curvature', 'change_curvature',
    # Eigenvalues (5)
    'eigenvalue_1', 'eigenvalue_2', 'eigenvalue_3',
    'sum_eigenvalues', 'eigenentropy',
    # Building scores (4)
    'verticality', 'horizontality', 'wall_score', 'roof_score',
    # Density (5)
    'density', 'local_density', 'num_points_2m',
    'neighborhood_extent', 'height_extent_ratio',
    # Architectural (5)
    'edge_strength', 'corner_likelihood', 'overhang_indicator',
    'surface_roughness', 'local_roughness',
]

Progress Bar Enhancements​

Updated: All feature computation modules

# Before: Basic progress
"Processing chunk X/Y..."

# After: Detailed context
"🎯 GPU Features [cuML] (10,234,567 pts, 42 chunks @ 245.2MB)"
"💻 CPU Features (5,123,456 pts, 21 chunks @ 122.6MB)"
"🔧 GPU Features [sklearn] (8,765,432 pts, 35 chunks @ 210.1MB)"

📈 Impact & Benefits​

Complete ML Training​

  • No Missing Features: All computed features now available for training
  • Better Model Performance: Access to all 35-45+ features improves model accuracy
  • Feature Analysis: Can now analyze importance of all computed features
  • Reproducibility: Feature names in metadata ensure consistent feature ordering

File Size Considerations​

  • NPZ/HDF5/PyTorch: ~3-4x larger (12 → 43 features)
  • LAZ: ~2-3x larger (7 → 35 extra dimensions)
  • Trade-off: Complete feature sets worth the storage increase

GIS Interoperability​

  • LAZ Export: All features as extra dimensions
  • QGIS/CloudCompare: Can visualize all computed features
  • Feature Inspection: Verify feature computation visually

🚀 Migration Guide​

Regenerate Datasets​

If you generated datasets before v2.4.2+, we recommend regenerating:

# Backup existing datasets
mv patches/ patches_old/

# Regenerate with full features
ign-lidar-hd process \
  --config-file examples/config_lod3_full_features.yaml \
  input_dir=data/ \
  output_dir=patches/

Verify Feature Completeness​

Check your datasets have all features:

import numpy as np

# Load a patch
data = np.load('patches/patch_001.npz')

# Check feature count and names
print(f"Features: {data['metadata'].item()['num_features']}")
print(f"Names: {data['metadata'].item()['feature_names']}")

# Expected for full mode: 38-43+ features
# Expected for LOD3: 35-38 features
# Expected for LOD2: 12 features

🔍 Verification​

Feature Export Test​

from ign_lidar.io.formatters.hybrid_formatter import HybridFormatter

# Create formatter
formatter = HybridFormatter(
    use_rgb=True,
    use_infrared=True,
    use_geometric=True
)

# Format a patch
formatted = formatter.format_patch(patch)

# Check metadata
metadata = formatted['metadata']
print(f"✓ Features exported: {metadata['num_features']}")
print(f"✓ Feature names: {metadata['feature_names']}")

LAZ Feature Inspection​

Open exported LAZ files in QGIS or CloudCompare:

  1. Load LAZ file
  2. View "Extra Dimensions" list
  3. Verify all 35+ geometric features present
  4. Visualize features using color mapping

📚 Documentation Updates​

🐛 Bug Fixes​

Critical Fixes​

  • Feature Export: All computed features now saved to disk (previously only 12/43)
  • LAZ Export: All geometric features now exported as extra dimensions
  • Progress Bars: Added missing context (point counts, chunk info, rates)

Impact​

  • High: Datasets missing up to 31 features in full mode
  • Medium: LAZ files missing 27+ extra dimensions
  • Low: Progress bars lacked helpful context

🎓 What's Next?​

See the Feature Modes Documentation for:

  • Complete feature lists for each mode
  • Performance comparisons
  • Best practices for feature selection
  • Training tips for LOD2/LOD3/Full modes

📝 Changelog Summary​

Added:

  • Feature name tracking in metadata
  • Feature count in metadata
  • Enhanced progress bars with detailed statistics
  • Complete feature export across all formats

Changed:

  • Feature matrix construction exports ALL features
  • LAZ export includes ALL geometric features
  • Progress bars show point counts, chunks, memory, rates
  • File sizes increase ~3-4x for complete feature sets

Fixed:

  • Feature export bug (only 12/43 features saved)
  • LAZ missing geometric features
  • Progress bars missing helpful context

Ready to use complete feature sets? Get Started →