Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | [wip] minor tweaks, resume development |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | master | trunk |
| Files: | files | file ages | folders |
| SHA3-256: |
19689689ea94b8d2afff611fd8389b20 |
| User & Date: | jack 2024-05-14 08:13:53 |
Context
|
2024-05-14
| ||
| 10:45 | add a definition of done check-in: 7b66e5d647 user: jack tags: master, trunk | |
| 08:13 | [wip] minor tweaks, resume development check-in: 19689689ea user: jack tags: master, trunk | |
|
2023-05-17
| ||
| 16:07 | Clear separation of trainining passes (resolves fixme) check-in: 10fe2857c0 user: jack tags: master, trunk | |
Changes
Changes to Code/dataset-and-aesthetics.lisp.
| ︙ | ︙ | |||
88 89 90 91 92 93 94 |
(funcall ,cont ,@lambda-list))
`(lambda (,cont ,raw-point)
(let ((,point-values (point-values ,raw-point)))
(funcall ,cont ,@lambda-list))))))
(values (compile nil fun) fun)))))
;;; MAYBE currently we access dataset-head and then map points and use
| | | 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
(funcall ,cont ,@lambda-list))
`(lambda (,cont ,raw-point)
(let ((,point-values (point-values ,raw-point)))
(funcall ,cont ,@lambda-list))))))
(values (compile nil fun) fun)))))
;;; MAYBE currently we access dataset-head and then map points and use
;;; point-values. Perhaps a better approach will be passing header and element
;;; accessor as functions.
(defun bind-data (aesthetics dataset &key (class <mapped-data>))
(make-instance class :dataset dataset :aesthetics aesthetics))
;;; Binding data conses a new vector so it may be potentially space and time
;;; consuming operation. We hide that behind the protocol so an alternate
;;; appraoch is possible.
|
| ︙ | ︙ |
Changes to Code/plot.lisp.
| ︙ | ︙ | |||
23 24 25 26 27 28 29 |
(dolist (layer (layers plot))
#+dont-retrain-layers
(unless (trained-dataset layer)
;; good for rendering, bad for mutations
(train-layer plot layer))
(train-layer plot layer)))
| | | 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
(dolist (layer (layers plot))
#+dont-retrain-layers
(unless (trained-dataset layer)
;; good for rendering, bad for mutations
(train-layer plot layer))
(train-layer plot layer)))
(defun train-plot (plot)
(dolist (layer (layers plot))
(compute-facets plot layer)
(do-facets (facet plot)
(let* ((dataset (dataset facet))
(scales (scales facet))
(aesthetics (layer-aesthetics plot layer))
(stat-tr (statistical-transformation layer))
|
| ︙ | ︙ | |||
54 55 56 57 58 59 60 |
(c:define-presentation-method c:present ((plot <polyclot>)
(type <polyclot>)
stream
(view <polyclot-view>)
&rest args)
(declare (ignore args))
(with-theme (plot)
| > > | > | | | | | < | | | | > > > > | | 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 |
(c:define-presentation-method c:present ((plot <polyclot>)
(type <polyclot>)
stream
(view <polyclot-view>)
&rest args)
(declare (ignore args))
(with-theme (plot)
(let ((offset-x 0)
(offset-y 0))
;;c:formatting-item-list (t :x-spacing 20 :y-spacing 10 :n-columns 2)
(do-facets (facet plot)
(c:with-translation (stream offset-x offset-y)
(let ((coords (coords facet))
(scales (scales facet)))
(progn ;c:formatting-cell (*standard-output*)
(c:draw-rectangle* *standard-output* 0 0 1 1
:ink c:+grey+ :filled nil)
(with-coords (coords scales)
(dolist (layer (layers plot))
(render-geom layer coords (trained-dataset facet layer))))
(render-guides plot coords))))
(if (zerop offset-x)
(incf offset-x)
(setf offset-x 0
offset-y 1))))))
(defun polyclot (*plot* *standard-output*)
(train-plot *plot*)
(c:present *plot* <polyclot>
:view (<polyclot-view>)
:sensitive nil
:allow-sensitive-inferiors t))
|
Changes to Code/theme.lisp.
| ︙ | ︙ | |||
38 39 40 41 42 43 44 | ;;; FIXME can we leverege McCLIM space requirement and layout protocols? (defgeneric call-with-theme (theme cont)) ;;; Estabilishes a context for drawing at the current cursor position as a unit ;;; box [0 0 1 1] and adds the plot decorations. (defmethod call-with-theme ((theme <theme>) cont) | < | | | | < < | < | | | | 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
;;; FIXME can we leverege McCLIM space requirement and layout protocols?
(defgeneric call-with-theme (theme cont))
;;; Estabilishes a context for drawing at the current cursor position as a unit
;;; box [0 0 1 1] and adds the plot decorations.
(defmethod call-with-theme ((theme <theme>) cont)
(when (climi::null-bounding-rectangle-p theme)
(multiple-value-bind (cx cy) (c:stream-cursor-position *standard-output*)
(setf (c:rectangle-edges* theme)
(values cx cy (+ cx (width theme)) (+ cy (height theme))))))
(c:with-bounding-rectangle* (x0 y0 :width width :height height) theme
(c:with-translation (t x0 y0)
(c:with-scaling (t width height)
(funcall cont)))))
(defmacro with-theme ((theme) &body body)
`(call-with-theme ,theme (lambda () ,@body)))
|
Changes to README.org.
1 2 3 4 5 | #+title: Polyclot #+author: Daniel Kochmański * Introduction | | | | < | | < | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | #+title: Polyclot #+author: Daniel Kochmański * Introduction Polyclot is a CLIM library for plotting data inspired by ggplot2 and the article "A Layered Grammar of Graphics" by Hadley Wickham. Additionally we extend it with a possibility to interact and incrementally modify plots. * Description A single plot is composed of: - a default dataset and its mapping to aesthetics - one or more layers each having a geom - one scale for each aesthetic - a statistical transformation - a coordinate system - a facet specification Additionally non-data parts are specified by the theme. The theme defines properties of the plot like its size, margins, the title, colors, etc. Faceting allows grouping data into separate datasets that are displayed in separate "boxes". There are three types of facets: none, wrap and grid. Coordinate system maps the position of objects onto the plane of the plot. Facets will have different coordinate system, but local coordinates for drawing are always a rectangle (0 0 1 1). Scales are used to map a data point to its aesthetic attribues. Each used aesthetic has exactly one scale associted. There are three types of scales: - position scales - guides are axes - color scales - guides are legends - other scales - guides are legends * Developer Manual |
| ︙ | ︙ | |||
82 83 84 85 86 87 88 89 90 | |----------+-------------+------------------------------------------------------| | labels | :label | text on the plot | | hjust | :align-x | text alignment (:left :center :right) | | vjust | :align-y | text alignment (:top :center :baseline :bottom) | * Development notes :noexport: ** Progress | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 |
|----------+-------------+------------------------------------------------------|
| labels | :label | text on the plot |
| hjust | :align-x | text alignment (:left :center :right) |
| vjust | :align-y | text alignment (:top :center :baseline :bottom) |
* Development notes :noexport:
** Architecture
The central piece is the instance of the class <polyclot>. It has the following
slots:
- dataset :: raw dataset [fixme] document dataset-head, dataset-body, streams
- aesthetics :: default mapping between data and aesthetics (inherited [layers])
- layers :: each layer is a serie of data that is placed on the plot (geoms)
- scales :: scales are computed based on layers, and then they transform data to
match the desired viewport and the data type
- coords :: the overall coordiante system for the plot (i.e cartesian)
- faceting :: grouping of data into multiple plots based on some characteristic
- theme :: theme adds all the fluff around actual plot and sets initial coords
A newly created <polyclot> object, before being plotted, needs to be trained.
This is because we don't know anything yet about the data -- for example its
range is unknown. Since data is faceted, we need to split the data first. The
first step that goes over all data is called training:
#+begin_src lisp
(defun train-plot (plot)
(map nil #'train-layer (layers plot)))
(defun train-layer (plot layer)
(compute-facets! plot layer)
(map nil #'train-facet (facets plot)))
(defun train-facet (plot facet)
(analyze-data)
(update-facet))
#+end_src
After the plot is trained, we can proceed to drawing it. To draw a <polyclot>
object on the stream it is enough to call ~(polyclot *plot* *clim-stream*)~.
Rendering of the plot works as follows:
1. The theme is used to show decorations, titles and legends
2. The theme estabilishes coordinates for the whole plot [0 0 1 1]
3. Each facet estabilishes coordinates for the individual facet and:
a. renders all layers using the trained dataset for the facet
b. renders guides
Rendering layers and guides depends on the ~coords~ and ~scales~. Both objects
depend on the facet.
** Progress
As of [2024-05-08 Wed] we have:
- coord: cartesian
- geoms: point, line and bar
- stats: identity and count
- adjut: jitter
- scales: {continuous, discrete, binned} x {x, y}
- facets: none, wrap, grid
Dataset is for now parsed from a list (car contains headers) into an instance of
~<mapped-data>~, where data-array is an extensible vector of ~<row>~ instances.
Aesthetics are slots in each ~<row>~ - this isn't efficient nor extensible.
The API for mapping aesthetics is IMO good as it is now (and provides sane
defaults), i.e: ~:aesthetics (list :x "x" :y 3 :ink #'random-ink)~. I think
|
| ︙ | ︙ | |||
124 125 126 127 128 129 130 | - [X] <cartesian> defaults to the screen orientation (should be graphics) - [ ] <row> is an instance with fixed slots, we want to extend it (a vector?) - [-] all geoms are very primitive often not even using relevant aes - [X] <geom-point> - [X] <geom-line> - [X] <geom-bar> - [ ] provide extra polishing(!) | | | | | 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 |
- [X] <cartesian> defaults to the screen orientation (should be graphics)
- [ ] <row> is an instance with fixed slots, we want to extend it (a vector?)
- [-] all geoms are very primitive often not even using relevant aes
- [X] <geom-point>
- [X] <geom-line>
- [X] <geom-bar>
- [ ] provide extra polishing(!)
- [7/9] implementation of scales and guides
- [X] basic framework for scales is in place already
- [X] scale-data :: transform the dataset (i.e log)
- [X] train-scales :: combine ranges of all datasets in the plot
- [X] continuous scale
- [X] discrete scale
- [X] binned scale
- [X] implement for aesthetics other than x/y
- [X] ink
- [X] line-style
- [ ] when we work with discrete/binned positions we add .5 :: this does not
seem to be correct, consider adjust-position dodge - it must fit numerous
elements between [n; n+1], while automatic translation .5 fits only one
element; should this translation be performed even by position-identity?
- [ ] handle n/a values
- [1/4] position adjustment
- [X] dodge bars
|
| ︙ | ︙ | |||
190 191 192 193 194 195 196 197 198 199 200 201 202 203 | News: - [2022-11-08 Tue] moved the repository fossil.turtleware.eu/polyclot - [2022-11-16 Wed] implemented binned scale (and fixed scale mapping) - [2022-12-11 nie] improvements to scales, a simple listener tool (demo) - [2022-12-13 wto] add colors, initial render of the color scale - [2022-12-14 śro] position adjustment (dodge) - [2023-05-17 Wed] the architecture has been changed to account for faceting ** Faceting Facets are being implemented after a single plot abstraction has been defined and that demands some architectural changes. Let's start with a description: Faceting is an operation of splitting datasets into a few separate sets that are | > | 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 | News: - [2022-11-08 Tue] moved the repository fossil.turtleware.eu/polyclot - [2022-11-16 Wed] implemented binned scale (and fixed scale mapping) - [2022-12-11 nie] improvements to scales, a simple listener tool (demo) - [2022-12-13 wto] add colors, initial render of the color scale - [2022-12-14 śro] position adjustment (dodge) - [2023-05-17 Wed] the architecture has been changed to account for faceting - [2024-05-08 Wed] resuming development (almost a year! In train after els'24) ** Faceting Facets are being implemented after a single plot abstraction has been defined and that demands some architectural changes. Let's start with a description: Faceting is an operation of splitting datasets into a few separate sets that are |
| ︙ | ︙ | |||
211 212 213 214 215 216 217 | ** Incremental plots Ideally we'd like to be able to add new data to the existing plot. Consider: - adding a new data point may change scales and influence whole facet - it should be possible to add a set of new points without retraining repeatedly - it would be cool to have some a GC possibility (i.e "remove older than") - the order of data points matters (consider a line plot) | < | 256 257 258 259 260 261 262 263 264 265 266 267 268 269 | ** Incremental plots Ideally we'd like to be able to add new data to the existing plot. Consider: - adding a new data point may change scales and influence whole facet - it should be possible to add a set of new points without retraining repeatedly - it would be cool to have some a GC possibility (i.e "remove older than") - the order of data points matters (consider a line plot) ** Scales When the plot is processed then each aesthetic has an associated a single scale shared by all layers. Generally there are three major types of scales: | scale type | input type | source range | range | result | |
| ︙ | ︙ |