diff --git a/_misc/md_preview.pl b/_misc/md_preview.pl new file mode 100755 index 0000000..0dea0fa --- /dev/null +++ b/_misc/md_preview.pl @@ -0,0 +1,36 @@ +#!/usr/bin/env perl +use warnings; +use strict; +use open qw/:std :utf8/; +use FindBin (); + +# Generate a preview of the site using `markdown` +# (I use this mostly just to check for any Markdown syntax mistakes) +#TODO Later: Use a markdown processor that handles GitHub's markdown enhancements? + +my $dir = $FindBin::Bin.'/..'; +opendir my $dh, $dir or die $!; +my @files = grep { ! -d } map { "$dir/$_" } sort grep {/\.md\z/i} readdir $dh; +close $dh; + +print <<'ENDHTML'; + + + + +WebPerl Site Preview + + +ENDHTML + +print "
\n"; +for my $f (@files) { + system('markdown',$f)==0 + or die "markdown failed, \$?=$?"; + print "
\n"; +} + +print <<'ENDHTML'; + + +ENDHTML diff --git a/building.md b/building.md index 3671e72..4e4a285 100644 --- a/building.md +++ b/building.md @@ -9,7 +9,145 @@ Building - =================== -Nothing here just yet - come back soon! +Prerequisites +------------- + +- Linux, a fairly modern release is strongly recommended, and `bash` + (tested on a minimum of Ubuntu 16.04) +- `git` (e.g. Ubuntu/Debian: `sudo apt-get install git-core`) +- Build tools for `perl`; for example on Ubuntu: + `sudo apt-get install build-essential` and `sudo apt-get build-dep perl` +- Perl, at least v5.26 (for example via [Perlbrew](http://perlbrew.pl/)) +- [Emscripten](http://emscripten.org) SDK, 1.38.10 and up, + please see the prerequisites and installation instructions at + +- The build script has several CPAN dependencies. One way to install them + is using [lazy](https://metacpan.org/pod/lazy): first, + install "lazy", then run e.g. `perl -Mlazy build.pl --help`. + Otherwise, the modules used by `build.pl` can be seen in + [its source](https://github.com/haukex/webperl/blob/master/build/build.pl) + grouped near the top of the file. +- A working Internet connection is needed for installation and the first build. + + +Source Code +----------- + +The source code is in two repositories: + +- - the main WebPerl repository + +- - a fork of the Perl 5 source + repository where the WebPerl-specific patches are applied + +You only need to check out the first of the two, the `emperl5` repository +is checked out by the build script. + + +Running the Build +----------------- + +1. Fetch the source code. + + $ git clone https://github.com/haukex/webperl.git + $ cd webperl + +2. Install the [prerequisites](#Prerequisites). + +3. Edit the configuration file, `./build/emperl_config.sh`, to fit + your system. For a first build, just make sure the path to + `emsdk_env.sh` is correct. + +4. Source the configuration file to set the environment variables. + Remember to do this anytime you change variables. You may also + add the sourcing of the configuration file to your `~/.bashrc`. + + $ . ./build/emperl_config.sh + +5. Run the build script: + + $ build/build.pl + +6. If the build succeeds, the output files `emperl.*` will be + copied to the `web` directory of the repository. You can + then use the files in the `web` directory as described in + [Using WebPerl](using.html). + + +Build Process Overview +---------------------- + +The build script `build.pl` tries to take care of as much of the build process as +possible. Most of the work happens in a subdirectory `work` of the repository. +Similar to `make`, it tries to not run build steps that don't need to be rerun. + +> A brief note on naming: +> +> - *`emperl`* is generally used for the build products of Emscripten +> - *`emperl5`* is the Perl 5 source tree modified for WebPerl +> - *WebPerl* is the finished product, including `emperl` +> and the WebPerl APIs (`WebPerl.pm` and `webperl.js`) + +The steps in the build process are roughly as follows. +Since WebPerl is still in beta, they are subject to change. +See +[the source of the `build.pl` script](https://github.com/haukex/webperl/blob/master/build/build.pl) +for the current details. + +1. Patch Emscripten + (currently just a minor patch, but important for Perl) + +2. Fetch/update the `emperl5` Perl source tree + +3. If necessary, build "host Perl" - in Perl's cross-compilation system, + this is the Perl that is built for the host system architecture, + i.e. in the case of Linux, a normal build of Perl for Linux. The + `miniperl` from the host Perl will be used for some of the build + steps for the target architecture. + (Note: This step can take quite a while, but it usually only needs + to be run once.) + +4. Download and extract any CPAN modules, such as the required `Cpanel::JSON::XS`, + into the Perl source tree so that they will be built as part of the normal + build process and any XS extensions linked statically into the `perl` binary. + (See ["Adding CPAN Modules"](#adding-cpan-modules)) + +5. Run Perl's `Configure` script using the custom "hints" file for the Emscripten + architecture. + +6. Run `make` to compile `perl`. This produces a file `perl.bc` with LLVM IR + bitcode, which the Emscripten compiler will then compile to JavaScript/WebAssembly. + Because some steps in the build process require a working `perl` binary, + Emscripten's compiler is used together with a supporting JavaScript file to + generate JavaScript code that can be run with `node.js` (called `nodeperl_dev.js`). + +8. Run the equivalent of `make install`, which copies all the Perl modules + etc. into the target directory that will become part of the Emscripten + virtual file system. Then, we clean this directory up by deleting anything + that we don't need for WebPerl: additional binaries (it's a single-process + environment), `*.pod` files, as well as stripping the POD out of `*.pm` + files, etc. to reduce the download size. + +9. The Emscripten compiler is used to take the previously compiled `perl.bc` + and build the final output, `emperl.js` along with the corresponding + `.wasm` and `.data` file. + + +Adding CPAN Modules +------------------- + +In the configuration file `emperl_config.sh`, the variable `EMPERL_EXTENSIONS` +is a whitespace-separated list of module names. `build.pl` will fetch these +from CPAN and extract them into the `ext` directory of the Perl source tree +so that they are compiled along with Perl. Any XS modules that need to be +linked into `perl` need to be added to the variable `EMPERL_STATIC_EXT` in +the format expected by Perl's `static_ext` configuration variable, +so for example `Cpanel/JSON/XS` instead of `Cpanel::JSON::XS` +(see ). + +Note that the build script does **not** automatically fetch modules' +dependencies, for now you will need to resolve them and add them to +`EMPERL_EXTENSIONS` yourself. (This may be improved upon in the future.) *** @@ -17,3 +155,11 @@ Nothing here just yet - come back soon! Additional notes on building WebPerl may be found in the [GitHub Wiki](https://github.com/haukex/webperl/wiki/Building-WebPerl). +*** + +Copyright (c) 2018 Hauke Daempfling (haukex@zero-g.net) +at the Leibniz Institute of Freshwater Ecology and Inland Fisheries (IGB), +Berlin, Germany, + +Please see the ["Legal" page](legal.html) for details. + diff --git a/index.md b/index.md index 4348e42..8828c5d 100644 --- a/index.md +++ b/index.md @@ -10,23 +10,61 @@ WebPerl uses the power of [WebAssembly](https://webassembly.org/) and -[emscripten](http://emscripten.org/) to let you run Perl 5 in the browser! +[Emscripten](http://emscripten.org/) to let you run Perl 5 in the browser! -**Notice: WebPerl is in beta.** +**Notice: WebPerl is very much in beta.** Some things may not work yet, and parts of the API may still change. Your feedback is always appreciated! ```html ``` +- [**Download `webperl_prebuilt_v0.01-beta.zip`**](https://github.com/haukex/webperl/releases/download/v0.01-beta/webperl_prebuilt_v0.01-beta.zip) +- [**Get the sources on GitHub**](https://github.com/haukex/webperl) + + +Quick Start +----------- + +- Prerequisites: `perl` (a recent version is recommended, e.g. v5.26 and up), + [`plackup` from Plack](https://metacpan.org/pod/distribution/Plack/script/plackup), + and [Cpanel::JSON::XS](https://metacpan.org/pod/Cpanel::JSON::XS). + +- In a shell: + + $ wget https://github.com/haukex/webperl/releases/download/v0.01-beta/webperl_prebuilt_v0.01-beta.zip + $ unzip webperl_prebuilt_v0.01-beta.zip + $ cd webperl_prebuilt_v0.01-beta + $ plackup webperl.psgi + HTTP::Server::PSGI: Accepting connections at http://0:5000/ + +- Then point your browser at + or + + +You may also host the contents of the above ZIP archive on a webserver of your choice, +as described in [Using WebPerl](using.html). (Note: In `webperl_demo.html`, you'll +likely see "AJAX Failed!", which is to be expected since your webserver won't +know how to handle the example AJAX request.) + +Have fun! ️🐪 + + +*** + +Copyright (c) 2018 Hauke Daempfling (haukex@zero-g.net) +at the Leibniz Institute of Freshwater Ecology and Inland Fisheries (IGB), +Berlin, Germany, + +Please see the ["Legal" page](legal.html) for details. + diff --git a/notes.md b/notes.md index 0c21937..616163b 100644 --- a/notes.md +++ b/notes.md @@ -9,12 +9,14 @@ Notes - ========================= -TODOs ------ +To-Dos +------ - - - +- See also To-Dos in the source tree by grepping for `TODO` + or using the included `findtodo.sh`. Possible Improvements @@ -24,7 +26,9 @@ Possible Improvements - Test/Support sockets/WebSockets - for example, can we compile a DBD:: module to connect to a DB on the server? - A RPC module for communicating between client and server Perls -- Support some of the Emscripten API (like wget?) + - I think it's probably best to not have WebPerl prescribe a specific RPC mechanism, + since there's a big variety and many are pretty simple to implement using e.g. jQuery +- Support some of the Emscripten C API (like wget?) - Try to shrink the download size more (exclude more modules, ...?) @@ -39,8 +43,8 @@ Limitations - (`system` and `qx` support could theoretically be added by patching `pp_system`/`pp_backtick` in `pp_sys.c`) - No signals (except `SIGALRM`) - In the current configuration, `exit` is not supported, and therefore `atexit` handlers aren't supported - (see discussion in `webperl.js`, and `NO_EXIT_RUNTIME` in the Emscripten documentation - currently it - seems to make the most sense to build with `NO_EXIT_RUNTIME=1`) + (see discussion in [Using WebPerl](using.html), and `NO_EXIT_RUNTIME` in the Emscripten documentation - + currently it seems to make the most sense to build with `NO_EXIT_RUNTIME=1`) - Static linking, requires rebuild to add modules (Emscripten apparently only supports asm.js dynamic linking when dynamic memory growth is disabled, which is not very useful) @@ -54,3 +58,12 @@ Several people have built microperl with Emscripten: - Shlomi Fish - FUJI Goro + +*** + +Copyright (c) 2018 Hauke Daempfling (haukex@zero-g.net) +at the Leibniz Institute of Freshwater Ecology and Inland Fisheries (IGB), +Berlin, Germany, + +Please see the ["Legal" page](legal.html) for details. + diff --git a/using.md b/using.md index 4e32af6..ae325c4 100644 --- a/using.md +++ b/using.md @@ -9,7 +9,385 @@ ================ -Nothing here just yet - come back soon! +Basic Usage +----------- + +### Serving WebPerl + +You should serve WebPerl via a webserver of your choice, or you can +use the included simple `webperl.psgi` for testing. You can run it using +[`plackup` from Plack](https://metacpan.org/pod/distribution/Plack/script/plackup) +by simply saying `plackup webperl.psgi`. + +The following four files make up WebPerl: + +- `webperl.js` - Contains the WebPerl JavaScript API and supporting code. +- `emperl.js` - Emscripten-generated supporting JavaScript. +- `emperl.wasm` - The `perl` binary and libraries compiled to WebAssembly. +- `emperl.data` - The Emscripten virtual file system data (`.pm` files etc.). + +I strongly recommend you add a MIME type of `application/wasm` for `.wasm` files, +otherwise you may see warnings like +"wasm streaming compile failed: TypeError: Response has unsupported MIME type" and +"falling back to ArrayBuffer instantiation". +For example, in an Apache `.htaccess` file, you can say: `AddType application/wasm .wasm` + +Note that opening the files locally (via `file://`) will likely not work +due to browsers' Same-Origin Policy. + +See also the Emscripten deployment notes at +, +in particular I'd recommended using gzip encoding to serve the WebPerl files. + +### Including Perl code in your HTML + +In your HTML file, add the following (usually inside the `` tags): + + + +Then, you can add one or more `` tag which loads a +Perl script from the server - but not both! The code from multiple +`