SketchToRender is an architectural design-research prototype that transforms abstract facade sketches and patterns into multiple architectural facade interpretations.
The project does not aim to generate construction-ready designs. Instead, it supports early-stage architectural exploration by helping designers visualize how a sketch, rhythm, or pattern might be interpreted at building scale.
Architectural sketches are often layered, ambiguous, and exploratory. They are not final design documents — they are tools for testing ideas.
SketchToRender extends this process by:
- accepting an architectural sketch or facade pattern,
- preparing it as a ControlNet condition,
- preserving its main compositional rhythm,
- producing multiple facade alternatives,
- allowing the designer to compare results and return to the sketch with new decisions.
The system is designed as an iterative design-research tool rather than an automatic design replacement.
| Input | Facade sketch or pattern (PNG · JPG/JPEG · JFIF · WEBP · BMP · TIFF) |
| Condition Type | Threshold-based Scribble map |
| Base Model | Stable Diffusion 1.5 (local snapshot) |
| ControlNet | Scribble (lllyasviel/sd-controlnet-scribble) |
| LoRA | Parametric Facade LoRA — scale 0.60 |
| Scheduler | UniPC Multistep |
| Resolution | 512 × 512 |
| Steps / Guidance Scale | 30 / 6.0 |
| Control Strength / Guidance End | 0.85 / 0.55 |
| Output per Run | 6 seed-based facade variations |
| Hardware | CUDA-required (GPU) |
The current prototype focuses on:
- abstract facade patterns,
- front-elevation sketches,
- parametric facade rhythms,
- straight-on, multi-storey building interpretations.
The current implementation uses a single Scribble ControlNet pipeline. It is optimized to produce frontal facade interpretations rather than perspective views, standalone decorative panels, product renders, pavilions, roofs, sculptural shells, or generic three-dimensional masses.
Architectural Sketch / Pattern
↓
Image Loading and Validation
↓
Resize + Grayscale Conversion
↓
Threshold-Based Scribble Condition
↓
Stable Diffusion 1.5
+ Scribble ControlNet
+ Parametric Facade LoRA
↓
Seed-Based Batch Generation
↓
Images + Metadata + Logs
↓
Comparison and Design Evaluation
Below is one full run of the pipeline: a hand-drawn ink pattern, its ControlNet condition map, and the six resulting seed-based facade alternatives (seeds 42–47, control_strength=0.85, control_guidance_end=0.55, lora_scale=0.60).
Input
ControlNet Condition Map
Six Seed-Based Alternatives
Note: this run's condition map was produced by a color-aware ink-detection variant of the preprocessing step (recorded in its
run_meta.jsonascondition_type: "color_aware_scribble"), which differs from the plain threshold-based method documented above. That variant isn't yet part of this README's technical description — it will be added once the corresponding script is finalized and reviewed.
- Python
- PyTorch
- Hugging Face Diffusers
- Stable Diffusion 1.5
- Scribble ControlNet (
lllyasviel/sd-controlnet-scribble) - Parametric Facade LoRA
- PIL
- NumPy
- CUDA
- UniPC Multistep Scheduler
Note: Preprocessing in the current script is done entirely with PIL and NumPy. OpenCV is not used or required by this pipeline.
The application automatically resolves common image extensions:
- PNG
- JPG / JPEG / JFIF
- WEBP
- BMP
- TIFF
The input image is:
- validated,
- EXIF orientation-corrected,
- converted to RGB,
- composited over a white background when transparency exists,
- resized to the configured generation resolution.
The source sketch is converted into a binary, Scribble-style condition:
- convert to grayscale,
- resize to the target resolution,
- apply a threshold,
- convert the result back to RGB.
Current configuration:
SIZE = 512
THRESHOLD = 200
The system also records simple condition statistics — mean intensity, standard deviation, and dark-pixel density — which are stored in the run metadata for later comparison.
The generation pipeline combines:
- a locally available Stable Diffusion 1.5 model,
lllyasviel/sd-controlnet-scribble,- a parametric facade LoRA,
- a UniPC Multistep Scheduler.
Current baseline parameters:
STEPS = 30
GUIDANCE_SCALE = 6.0
CONTROL_STRENGTH = 0.85
CONTROL_GUIDANCE_END = 0.55
LORA_SCALE = 0.60
SEEDS = [42, 43, 44, 45, 46, 47]
A key design decision in this prototype is limiting Scribble ControlNet guidance to the early portion of the diffusion process:
CONTROL_GUIDANCE_END = 0.55
The intended behavior:
- during the early diffusion steps, the sketch controls the main composition and facade rhythm;
- during the remaining steps, Stable Diffusion has more freedom to develop architectural details such as windows, floor divisions, materials, depth, and entrances.
When control remains active throughout the entire process, the model tends to reproduce the input pattern too literally. Reducing the control duration helps balance sketch adherence, architectural readability, material interpretation, and facade detail generation.
The positive prompt encourages straight-on front elevations, complete multi-storey facades, visible floor divisions, window bays and openings, facade boundaries, realistic materials, subtle depth, and integration of the source pattern at building scale.
The negative prompt reduces unwanted interpretations such as decorative wall panels, flat pattern renders, product visualizations, perspective views, pavilions, shell structures, unrelated facade rhythms, and distorted or melted geometry.
Prompt content is fixed across a run so that output differences can be examined mainly through seed changes under otherwise identical settings.
The application generates six outputs per run — seeds 42 through 47 — using identical prompt and pipeline settings. Only the seed changes between outputs.
Using several seeds serves two purposes:
- exploring different architectural interpretations of the same sketch under fixed conditions;
- preserving reproducibility by recording the exact seed used for every output.
The goal is not to pick a random attractive image, but to compare how the same design input produces different facade characters under fixed production conditions.
Each execution creates a timestamped output directory:
outputs/
└── run_final_facade_YYYYMMDD_HHMMSS/
├── source_resized.png
├── condition_threshold200.png
├── a01_seed42.png
├── a01_seed42.json
├── a02_seed43.png
├── a02_seed43.json
├── ...
├── comparison_sheet.png
├── run_meta.json
└── run.log
run_meta.json stores run-level information: Python/PyTorch environment, CUDA status and GPU name, resolved input path, source and condition images, threshold value, condition statistics, prompt and negative prompt, generation parameters, base model, ControlNet model, LoRA path and scale, and the seed list.
Per-seed JSON files store, for each generated image: seed, image filename, input and condition references, prompt settings, ControlNet parameters, LoRA settings, and model information — making each output traceable and reproducible.
- CUDA availability validation (the script stops before model loading if no compatible CUDA environment is found)
- CPU-only PyTorch detection
- Readable error messages
- Image-format validation
- Exception handling for corrupted inputs
- Logging to both console and file
- VRAM usage logging
- Garbage collection between generations
- CUDA cache cleanup
- Model CPU offloading
- Optional xFormers acceleration, with attention-slicing fallback
- VAE slicing
Requirements
- CUDA-compatible NVIDIA GPU
- Python environment with CUDA-enabled PyTorch
- Compatible Diffusers installation
- Locally available Stable Diffusion 1.5 model
- Scribble ControlNet files
- Parametric facade LoRA file
Update the local paths before running:
BASE_MODEL = "path/to/local/stable-diffusion-1.5"
LORA_PATH = Path("path/to/pf_parametric_facade_v0.1.safetensors")
SKETCH_PATH = "sketch"Place an input image in the project directory, for example sketch.png, then run:
python SketchToRender_FINAL_Facade.pyThe script does not install or update packages during execution.
- How much of an abstract facade rhythm can be preserved?
- Which sketch characteristics lead to more coherent outputs?
- When do auxiliary lines become unintended architectural elements?
- How does control duration affect geometry and architectural detail?
- How much variation can be obtained through seed changes while retaining the original design idea?
- How can preprocessing reduce ambiguity in hand-drawn inputs?
The prototype performs better when the input contains:
- visible primary contours,
- a readable facade hierarchy,
- clear pattern density,
- limited overlapping helper lines,
- a front-elevation logic,
- distinguishable solid and void relationships.
It can generate multiple facade interpretations while retaining parts of the original rhythm and compositional structure.
- Dense helper lines may be interpreted as structural or facade elements.
- Highly ambiguous sketches can weaken facade coherence.
- Pattern scale may change between outputs.
- Generated images are conceptual visualizations, not construction-ready designs.
- The current version uses a single, threshold-based Scribble condition.
- Different sketch types may require different preprocessing strategies.
- Architectural quality still requires human evaluation and selection.
- The pipeline currently depends on local model paths and a CUDA environment.
- Configurable preprocessing profiles
- Separate modes for freehand sketches and facade patterns
- Automatic threshold estimation
- Additional condition types where technically justified
- User-controlled material and facade-language settings
- Structured experiment comparison
- Improved configuration management
- Command-line arguments
- Separation of configuration from source code
- Automated tests for input and metadata functions
- A lightweight user interface
The repository currently contains several development iterations. For portfolio and public use, the recommended structure is:
SketchToRender/
├── src/
│ ├── pipeline.py
│ ├── preprocessing.py
│ ├── generation.py
│ └── metadata.py
├── examples/
│ ├── input/
│ └── output/
├── docs/
├── requirements.txt
├── .gitignore
├── LICENSE
├── README.md
└── main.py
The current repository preserves earlier scripts as part of the development history. A future cleanup will consolidate the validated pipeline into a single, documented entry point.
SketchToRender is part of a broader portfolio exploring AI-assisted architectural visualization, computer vision, controlled image generation, multi-view architectural consistency, and design-research workflows.
The accompanying portfolio presents design intent, workflow diagrams, input/output comparisons, seed-based alternatives, technical decisions, and limitations and future work.
SketchToRender is a research prototype. Generated outputs:
- are not final architectural projects,
- are not construction documents,
- should not be used without architectural evaluation,
- are intended to support exploration, comparison, and design discussion.
Elif Kalender Architectural Visualization · AI-Assisted Design · Computer Vision