matt wilkiehttp://www.maphew.com/maphew+website@gmail.coma hewer of mapstag:www.maphew.com,2011-06-11:/atom/http://www.maphew.com/favicon.icohttp://www.maphew.com/feed-logo.png2015-05-11T07:00:00ZacrylamidExtract Invisible Attachmenttag:www.maphew.com,2015-05-11:/Fixes/extract-invisible-attachment2015-05-11T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>I periodically get email from ios users with invisible attachments. </p>
<blockquote>
<p><strong>TLDR:</strong> view raw, save base-64 to text, uudecode using <strong><code>certutil --decode</code></strong> (a native tool in Win7+)</p>
</blockquote>
<p>Last night I finally figured out what is going on: the attachment is marked as <em>"inline"</em> in the header, but the client (gmail in this case, but can happen with others too) doesn't render the attachment correctly. However since the media is inline it's not visible/extractable like regular attachments are.</p>
<h5>Symptoms</h5>
<ul>
<li>Email says there is in attachment (e.g. has a paperclip), but there is no place to click and save or download the attachment</li>
<li>broken image icon in message body</li>
</ul>
<p><img alt="" src="http://www.maphew.com/files/fixes/invis-attach-email.png"/> </p>
<h5>The fix</h5>
<ul>
<li>view original/raw message</li>
<li>Find <code>Content-Disposition: attachment; filename="some_filename.pdf"</code> and save everything between <code>Content-Id: <AAA888...></code> and <code>--Apple-Mail-AAA888...--</code> lines (don't include those lines) to a text file, e.g. <code>some_filename.uue</code></li>
<li>open command shell and run <code>certutil -decode some_filename.uue some_filename.pdf</code></li>
</ul>
<h6>Sources</h6>
<ul>
<li>Fowarded emails with attachments not showing up with paperclip icon, https://productforums.google.com/forum/#!topic/gmail/Mdujqj1zUic</li>
<li>Message with pdf attachment doesn't show as having an attachment, https://productforums.google.com/forum/#!topic/gmail/LbUIrEqzRXE</li>
<li>Super User: <a href="http://superuser.com/questions/213563/how-can-i-extract-an-attachment-from-the-email-body-encoded-as-base64">How can I extract an attachment from the email body (encoded as base64)</a></li>
<li>Dick on http://johnsad.ventures/software/base64encoderdecoder/ for the <em>certutil</em> tip</li>
</ul>The 1st modern Cahill-Keyes Map, maybetag:www.maphew.com,2013-12-18:/GIS/the-1st-modern-cahill-keyes-map-maybe2013-12-19T00:26:29Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>All maps distort things -- one or more of area, shape and distance. They [splatten] a roundish sphere into a flat piece of paper. Some distortions are much worse than others. The <a href="http://www.maphew.com\GIS\the-1st-modern-cahill-keyes-map-maybe\Projects\cahill-keyes-butterfly-map">Cahill-Keyes</a> projection is the best invention to date we humans have for minimizing the effects. Unfortunately very few know about it or have seen it. This map by Duncan Webb of Australia, possibly the first modern full scale version of the CK map projection could go a long way to rectify our common (mis)perception of the relative shapes and sizes of where we live. <em>If</em> it sees production -- you can help make that happen by pledging to buy one before January 3rd:</p>
<p><a href="http://www.indiegogo.com/projects/cahill-keyes-world-map-poster"><img alt="Duncan Webb's hardcopy Cahill-Keyes Map project" src="http://www.maphew.com/files/duncan-ck-map.png"/></a></p>
<p>What's all the fuss about? There are many resources out there explaining it more completely and eloquently than I can, but here's an info-graphic to get you started. Each of the pinkish blobs on the maps below <strong>occupy the same size and area of ground</strong>, but they sure don't look like it do they?</p>
<p><a href="http://www.maphew.com/files/Various_projections_with_error_ellipses.pdf"><img alt="Various map projections with tissot error ellipses" src="http://www.maphew.com/files/various_projections_with_error_ellipses.png"/></a></p>
<p><a href="http://www.maphew.com/files/Various_projections.pdf"><img alt="Various map projections" src="http://www.maphew.com/files/various_projections.png"/></a></p>
<p>[0]: <em>well, 'splatten' should be word darn it!</em></p>Oh Assign, Assign, wherefore art thou Assign?tag:www.maphew.com,2013-05-17:/Itches/oh-assign-assign-wherefore-art-thou-assign2013-05-17T07:54:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>Most of my computer life of the last few years has lived within Dropbox. It's proven to be a very hands off largely no-thought means of ensuring the 3 Windows computers I use regularly at work and home all have almost identical tool chains and environments. It's always bugged me though, that my "home" is buried under <code>C:\Users\matt\Dropbox</code> (though at least with Win7 it's not under that horrible "Documents and Settings" folder!). Way too freakin much typing just to get to the point of starting work, or fun.</p>
<p>I've tried several methods for making this more livable, and each succeeds to a certain extent -- though <em>none</em> come close to matching what we had 20 years ago with the <a href="http://en.wikipedia.org/wiki/AmigaOS" title="scan for assign">Amiga</a>.</p>
<h4>Move Dropbox to root of C:</h4>
<p>yech. user data on same partition as operating system. never been comfortable with this. I want OS and Programs over <em>[there]</em>, my docs and stuff <em>[here]</em>, and easily wipe or restore one without touching the other.</p>
<h4>Mapped "Network" Drive</h4>
<p>Open a command shell, fire off <code>net use B: \\localhost\c$\users\%username%\Dropbox /persistent:yes</code>.</p>
<p>(So nice that A: and B: almost never exist anymore, and are thus available, even when joined to a domain.)</p>
<p>Worked swimmingly well for several months, until learning that the mysterious Mecurial and Bazaar lock errors I would encounter at seemingly random occasions and took hours to get out of were due to timing issues in the underlying network transport protocols, even though everything was happening on the local machine and darn fast, to human perception.</p>
<p>Also can't use any commands requiring elevated privileges because "B:" isn't available to Administrator. (why o why doesn't Windows have <code>sudo</code>??)</p>
<h4>Subst-tituted Drive</h4>
<p><code>subst b: c:\users\%username%\dropbox</code></p>
<p>An effective cure for the timing issues noted previously. Suffers from lack of sudo, which has been real problem while trying to work on improving Leo and Osgeo4w install packages.</p>
<h4>Separate Partition</h4>
<p>The Dropbox app steadfastly refuses to use a bare drive as it's home, and insists on creating a folder called <code>Dropbox</code> where you point it. If you run a client older than 1.6 or so, before the database was encrypted, it is possible to open the <code>config.db</code> file in an sqlite3 aware db program and change the folder name manually, including to a bare drive letter. </p>
<p>There are some features lost with this, plus you'll also be syncing the <em>Recycle Bin</em> and <em>System Volume Information</em>, which could cause other issues.</p>
<h4>Mounted NTFS Folder</h4>
<p><strong>Finally</strong>, yesterday, I hit open the best possible solution short of the Amiga's Second Coming: create a separate partition and volume and instead of assigning it a drive letter mount it to an empty NTFS folder, and then <code>subst</code> that. </p>
<div class="highlight"><pre>C:\Users\<span class="o">%</span>username<span class="o">%</span>\Dropbox <span class="o"><--</span> mount volume here
subst b: C:\Users\<span class="o">%</span>username<span class="o">%</span>\Dropbox
</pre></div>
<p>For extra flourish, size the volume to your Dropbox limit. Doing so you always know at a glance how close you are to your quota. (Good idea to make it 15% than actual quota for breathing room. It's enough space to defrag, and Windows will get markedly slower on volumes with less free board.)</p>
<p><img alt="dropbox volume in My Computer" src="http://www.maphew.com/files/itches/dropbox-volume.jpg"/></p>Python fixit snippetstag:www.maphew.com,2013-05-16:/Python/python-fixit-snippets2013-05-16T17:54:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>Very short problems about python, found and fixed.</p>
<div class="highlight"><pre><span class="o">></span><span class="n">pip</span>
<span class="n">failed</span> <span class="n">to</span> <span class="n">create</span> <span class="n">process</span><span class="p">.</span>
</pre></div>
<p>whups, this worked yesterday. Whatup? --> drive letters/paths have been
changed. The path to python.exe specified in pip.exe has moved. Solution:
<a href="http://www.maphew.com/Python/install-pip-script/">reinstall pip</a> (and any other py programs similarly impacted).</p>Portable Leo from scratch on Windowstag:www.maphew.com,2013-04-28:/Projects/portable-leo-from-scratch-on-windows2013-04-28T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>Here is a recipe for installing <a href="http://leoeditor.com">Leo</a> from scratch on Windows (Win7 tested). The only prerequisites are command line <code>wget</code> in PATH and an internet connection.</p>
<p>About 50mb of files are downloaded and 200mb consumed when done.</p>
<p>Open a command prompt and run:</p>
<div class="highlight"><pre><span class="nb">mkdir</span> <span class="n">X:</span><span class="o">\</span><span class="n">testing</span>
<span class="n">pushd</span> <span class="n">X:</span><span class="o">\</span><span class="n">testing</span>
<span class="n">wget</span> <span class="o">-</span><span class="n">O</span> <span class="n">apt</span><span class="o">.</span><span class="n">exe</span> <span class="o">--</span><span class="nb">no</span><span class="o">-</span><span class="n">clobber</span> <span class="n">http:</span><span class="sr">//</span><span class="n">download</span><span class="o">.</span><span class="n">osgeo</span><span class="o">.</span><span class="n">org</span><span class="sr">/osgeo4w/x</span><span class="mi">86</span><span class="sr">/release/</span><span class="n">apt</span><span class="o">/</span><span class="n">apt</span><span class="o">-</span><span class="n">r1193M</span><span class="o">.</span><span class="n">exe</span>
<span class="n">SET</span> <span class="n">OSGEO4W_ROOT</span><span class="o">=</span><span class="nv">%</span><span class="err">~</span><span class="nv">dp0</span><span class="o">\</span><span class="n">root</span>
<span class="n">apt</span> <span class="n">setup</span>
<span class="n">apt</span> <span class="n">update</span>
<span class="n">apt</span> <span class="n">install</span> <span class="n">pyqt4</span> <span class="n">sip</span>
<span class="n">call</span> <span class="n">root</span><span class="o">\</span><span class="n">osgeo4w</span><span class="o">.</span><span class="n">bat</span>
<span class="n">wget</span> <span class="o">--</span><span class="nb">no</span><span class="o">-</span><span class="n">check</span><span class="o">-</span><span class="n">certificate</span> <span class="n">http:</span><span class="sr">//gis</span><span class="n">t</span><span class="o">.</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="sr">/maphew/</span><span class="mi">5393935</span><span class="sr">/raw/i</span><span class="n">nstall</span><span class="o">-</span><span class="n">pip</span><span class="o">.</span><span class="n">py</span>
<span class="n">python</span> <span class="n">install</span><span class="o">-</span><span class="n">pip</span><span class="o">.</span><span class="n">py</span> <span class="n">GO</span>
<span class="n">python</span> <span class="n">install</span><span class="o">-</span><span class="n">pip</span><span class="o">.</span><span class="n">py</span> <span class="n">GO</span>
<span class="n">pip</span> <span class="n">install</span> <span class="o">-</span><span class="n">i</span> <span class="n">https:</span><span class="sr">//</span><span class="n">testpypi</span><span class="o">.</span><span class="n">python</span><span class="o">.</span><span class="n">org</span><span class="o">/</span><span class="n">pypi</span> <span class="n">leo</span><span class="o">-</span><span class="n">editor</span>
<span class="n">python</span> <span class="n">apps</span><span class="o">\</span><span class="n">Python27</span><span class="o">\</span><span class="n">Scripts</span><span class="o">\</span><span class="n">leo</span>
</pre></div>
<p>After this the directory <code>X:\testing\root</code> can be renamed and/or moved anywhere.</p>
<p>To run leo in future sessions, simply call <code>x:\path\to\root\osgeo4w.bat python apps\Python27\Scripts\leo</code>, either as a batch file or windows shortcut.</p>
<p>IMPORTANT NOTE - this is just a proof of concept. It pulls together a couple of my different projects, and is very satisfying in that regard (yay! they work!), but is not ready for prime time. Feedback welcome.</p>
<p>(((this is a post in progress, see <a href="https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/leo-editor/portable|sort:date/leo-editor/pH4mHaSRbfc/tQcxjn37xKUJ">this thread</a> for more)))</p>Plus 1 for static site generators!tag:www.maphew.com,2013-04-20:/web-dev/plus-1-for-static-site-generators2013-04-20T10:28:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>There is no time like the present to really delve into and master static web generation:</p>
<p><em><a href="http://arstechnica.com/security/2013/04/huge-attack-on-wordpress-sites-could-spawn-never-before-seen-super-botnet/">Huge attack on WordPress sites could spawn never-before-seen super botnet. Ongoing attack from >90,000 computers is creating a strain on Web hosts, too</a></em></p>
<p><em><a href="http://arstechnica.com/security/2013/04/exclusive-ongoing-malware-attack-targeting-apache-hijacks-20000-sites/">Ongoing malware attack targeting Apache hijacks 20,000 sites. Mysterious "Darkleech" exposes visitors to potent malware exploits</a></em></p>
<p>I received the ArsTechnica newsletter with the above stories in my mailbox in the same 5 minute window that Google Webmaster Tools informed me that one of our websites for my work has likely been compromised. </p>
<p>No, it's not running Wordpress, but it is running a blog-style php platform that is several versions behind current because I didn't want to break the site while trying to upgrade a system I don't understand. Now this particular bit of procrastination has turned into a bite!</p>
<p>Guess I know what I'm doing for the next week or more. :-/</p>Install Pip scripttag:www.maphew.com,2013-04-16:/Python/install-pip-script2013-04-16T21:22:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p><em>Introducing a pure python means to install pip and it's requirements <a href="https://gist.github.com/maphew/5393935">all in one go</a>.</em></p>
<p>Installing <a href="http://www.pip-installer.org/en/latest/">pip</a> on Windows is relatively straightforward, now, but still a tad bumpy. Basically it consists of </p>
<ol>
<li>Download distribute or setuptools</li>
<li>Install it with <code>python distribute_setup.py</code></li>
<li>Download the get-pip script</li>
<li>Install with <code>python get-pip.py</code></li>
</ol>
<p><em>(Aside: curiously the <a href="https://pypi.python.org/pypi/distribute/">distribute docs</a> say you can use pip to install distribute!)</em></p>
<p>On Windows this means installing <code>wget</code> or <code>curl</code> in order to follow the install instructions as given, or pointing and clicking your through <em>click-this-url >> save as >> choose a spot >> open a python shell >> ...etc.</em>. Easy, but boring and easy to make small mistakes which means repeating all or part of the recipe. </p>
<p>So as part of my continuing quest to learn enough to be called a true pythonista, I wrote a small script to do it all in one go, <a href="https://gist.github.com/maphew/5393935">install-pip.py</a>. It is adapted from <a href="http://stackoverflow.com/users/585592/fernando-macedo">Fernando Macedo</a>'s answer in <a href="http://stackoverflow.com/questions/4750806/how-to-install-pip-on-windows/">How to install pip on windows?</a>. The seminal idea is it use <code>urllib</code> to do the download, which is included in Python's standard library -- isn't it just great to have batteries included? (though it would begs the question of why there isn't a python package installer battery included...)</p>
<p>The script does a bit more, but here's the essential idea (the blog renderer is messing up the indentation, sorry):</p>
<div class="highlight"><pre><span class="n">sources</span> <span class="p">=</span> <span class="p">[</span>
<span class="s">'http://python-distribute.org/distribute_setup.py'</span><span class="p">,</span>
<span class="s">'https://raw.github.com/pypa/pip/master/contrib/get-pip.py'</span><span class="p">,</span>
<span class="p">]</span>
<span class="k">for</span> <span class="n">src</span> <span class="n">in</span> <span class="n">sources</span><span class="p">:</span>
<span class="n">f</span> <span class="p">=</span> <span class="n">urlopen</span><span class="p">(</span><span class="n">src</span><span class="p">).</span><span class="n">read</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span> <span class="n">exec</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">except</span><span class="p">:</span> <span class="n">pass</span>
</pre></div>
<p>Passing on an exception instead of dealing with it isn't a great idea, maybe even a stupid one, but without it only the first file is executed. Comments, fixes, and enhancements are all welcome. :) </p>
<p>See <a href="https://gist.github.com/maphew/5393935">https://gist.github.com/maphew/5393935</a></p>Deploy website with rsynctag:www.maphew.com,2013-03-17:/web-dev/deploy-website-with-rsync2013-03-17T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p><em>Recipe for copying my Acrylamid-generated static html website to my webhost (<a href="http://www.nearlyfreespeech.net">nearlyfreespeech.net</a>).</em></p>
<p>In a nutshell, I use <a href="http://qtdtools.doering-thomas.de/page.php?seite=0&sub=0&lang=en">QtdSync</a> to generate the basic rsync update script, and then add rsync advanced options as make sense to me. From QtdSync's progress screen the rsync command line can be copied and pasted to create a batch file (strip the <code>-e ... sshpass...</code> line for interactive password prompting).</p>
<p>QtdSync is a bit twitchy around the <em>"backup/don't backup into subfolder"</em> options. Sometimes they appear to have the opposite effect of what they say. I still found it much easier than trying to construct the appropriate rsync script from scratch myself, just test on a small data set until the settings are ironed out. The backup set <code>.qtd</code> files are straightforward XML so it's pretty easy to check the results of your settings.</p>
<h5>QtdSync hard coded rsync options</h5>
<p>These are always used.</p>
<p><code>-rltv</code> which means:<br />
<code>--recursive</code><br />
<code>--links</code> - <em>copy symlinks as symlinks</em><br />
<code>--times</code> - <em>preserve modification times</em><br />
<code>--verbose</code></p>
<p><code>--progress</code> - <em>show progress</em></p>
<h5>QtdSync editable advanced rsync options</h5>
<p><code>--delete</code> - <em>delete extraneous files from destination dirs</em><br />
<code>--force</code> - <em>force deletion of directories even if not empty</em> <br />
<code>--ignore-errors</code> - <em>delete even if there are I/O errors</em><br />
<code>--hard-links</code> - <em>preserve hard links</em></p>
<h5>Extended rsync options</h5>
<p>Add these in the <em>Rsync Expert Settings</em> pane. When added as Task they override the global settings.</p>
<p><code>--compress</code> - <em>compress file data during the transfer</em>, saves bandwidth, speeds up transfers, 'nuff said</p>
<p><code>--skip-compress=%_ignore_ext%</code> - <em>skip compressing files with a suffix in LIST</em>, not yet using, but seems like a good idea</p>
<p><code>--checksum</code> - <em>skip based on checksum, not mod-time & size</em>, because preserving timestamps between Windows and 'nix doesn't always work. I suspect it's faster to use <code>--modify-window=NUM</code> with a window like 2secs (that's what <a href="http://xxcopy.com"><code>xxcopy /ff</code></a> uses) but I'm not that concerned with performance and checksum should be more reliable.</p>
<p>For these next two, if something goes wrong the website will stay in previous state. Not quite the same as atomic updates in a db but close enough for me.<br />
<code>--delay-updates</code> - <em>put all updated files into place at transfer's end</em><br />
<code>--delete-delay</code> - <em>find deletions during, delete after</em></p>
<p><code>--human-readable</code> - <em>output numbers in a human-readable format</em></p>Going Acrylictag:www.maphew.com,2013-03-16:/web-dev/going-acrylic2013-03-16T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p><em>Recipe used for converting httrack snapshot of Mindtouch wiki to markdown for acrylamid</em></p>
<h4>Generate the processing list</h4>
<div class="highlight"><pre><span class="n">dir</span> <span class="o">/</span><span class="n">s</span><span class="o">/</span><span class="n">b</span> <span class="o">\</span><span class="n">www</span><span class="o">\</span><span class="n">maphew</span><span class="p">.</span><span class="n">com</span><span class="o">\*</span><span class="p">.</span><span class="n">html</span> <span class="o">></span> <span class="n">process</span><span class="o">-</span><span class="n">list</span><span class="p">.</span><span class="n">xt</span>
</pre></div>
<p>Edit process-list and remove junk, fix bad filenames (resultant from double quotes in name). </p>
<p>Copy to excel and:</p>
<ul>
<li>convert text to table using <code>\</code> for delim, </li>
<li>apply conditional formatting highlighting <code>.html</code></li>
<li>sort by html column</li>
<li>remove duplicates</li>
<li>save tab delimited</li>
<li>search and replace tabs with <code>\</code>, removing dupes</li>
</ul>
<h4>Scripted Html to Markdown to Acrylamid</h4>
<div class="highlight"><pre><span class="k">for</span> <span class="o">/</span><span class="n">f</span> <span class="c">%a in (process-list-cleaned.txt) do @mkdir .%~pa</span>
<span class="k">for</span> <span class="o">/</span><span class="n">f</span> <span class="c">%a in (process-list-cleaned.txt) do ^</span>
<span class="p">@</span><span class="n">pandoc</span> <span class="o">--</span><span class="n">to</span> <span class="n">markdown</span> <span class="o">--</span><span class="n">standalone</span> <span class="o">--</span><span class="n">template</span> <span class="n">acrylamid</span><span class="o">-</span><span class="n">pandoc</span><span class="o">-</span><span class="n">template</span><span class="p">.</span><span class="n">txt</span> "<span class="c">%a" -o ".\%~pa\%~na.md"</span>
<span class="n">acrylamid</span> <span class="n">init</span> <span class="n">converted</span>
<span class="n">rd</span> <span class="o">/</span><span class="n">s</span><span class="o">/</span><span class="n">q</span> <span class="n">converted</span><span class="o">\</span><span class="n">content</span>
<span class="n">move</span> <span class="n">www</span><span class="o">\</span><span class="n">maphew</span><span class="p">.</span><span class="n">com</span> <span class="n">converted</span>
<span class="n">pushd</span> <span class="n">converted</span>
<span class="n">rename</span> <span class="n">maphew</span><span class="p">.</span><span class="n">com</span> <span class="n">content</span>
<span class="n">rd</span> <span class="o">/</span><span class="n">s</span><span class="o">/</span><span class="n">q</span> <span class="p">.</span><span class="o">.\</span><span class="n">www</span>
<span class="n">copy</span> <span class="o">\</span><span class="n">www</span><span class="o">\</span><span class="n">acr</span><span class="o">\</span><span class="n">confy</span><span class="p">.</span><span class="n">py</span> <span class="o">.\</span><span class="n">conf</span><span class="p">.</span><span class="n">py</span>
<span class="n">xcopy</span> <span class="o">\</span><span class="n">www</span><span class="o">\</span><span class="n">acr</span><span class="o">\</span><span class="n">theme</span><span class="o">\*</span> <span class="n">theme</span><span class="o">\*</span>
<span class="n">acrylamid</span> <span class="n">compile</span> <span class="o">--</span><span class="n">search</span>
<span class="p">::</span> <span class="nb">fix</span> <span class="n">title</span> <span class="n">collisions</span> <span class="n">as</span> <span class="n">needed</span>
</pre></div>
<h4>Clean up header & footer crud</h4>
<p>Remove Mindtouch leftovers such as <em>"javascript must be enabled"</em> and <em>"Powered by..."</em>, etc. etc. by using search and replace across all open files.</p>
<p>Vim regexes:</p>
<div class="highlight"><pre><span class="p">:</span><span class="n">bufdo</span><span class="p">:</span><span class="c">%s/^This application requires Javascript to be enabled\.\_.* Table of contents$//</span>
<span class="p">:</span><span class="n">bufdo</span><span class="p">:</span><span class="c">%s/\*No headers\*//</span>
<span class="p">:</span><span class="n">bufdo</span> <span class="c">%s/Powered by \[MindTouch Core\_.*maphew)//e</span>
<span class="p">:</span><span class="n">bufdo</span> <span class="c">%s/---\n\n\n*/---\r\r/e</span>
<span class="p">:</span><span class="n">bufdo</span> <span class="c">%s/$title:\(.*\) - maphew$/title:\1/e</span>
</pre></div>
<p>Sources:</p>
<ul>
<li>http://vim.wikia.com/wiki/Search_across_multiple_lines</li>
<li>http://vim.wikia.com/wiki/Search_and_replace_in_multiple_buffers</li>
</ul>
<h5>Fix dates</h5>
<p><strong>This marks the end of the automated repeatable process.</strong></p>
<p>Read through <em>Special_RecentChanges</em> and manually edit each .md to reflect last time touched. We don't have dates for content which was migrated to Mindtouch (prior to 17.10.2009).</p>
<h4>Add tags</h4>
<p>The folders are the primary tag, so search *.md and sort by location, then drag'n'drop each group into a handy text editor and paste the appropriate tag(s) in each. </p>
<p>I'm sure there's a way to automate this, but I decided it would be faster to brute force my way through (esp. since we've already broken with repeatable process anyway).</p>Spring, Cyclestag:www.maphew.com,2013-03-01:/category/spring-cycles2013-03-01T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>Well, here I am once again looking at the detritus of my website and wondering what to do with it. Each page represents, at least, an hour or three of endeavour. Many have no relevance any more and or of little if any interest to anyone, including me. Others though... well lets just say I'm not quite ready to pull the plug and let them drain away.</p>
<p>While not a regular experience, this is a recurring one. We've been here before. Each time I've devoted a few days or months to rewriting or transforming old content into a new container, add some new content along the way. From static html to wiki, to a cms, to another wiki, back to static html, ...to whatever this current cycle results in.</p>
<p>One of the historical features of this cycle I'm noting today is: after the phoenix dies and is resurrected, the life soon fades. The form remains, because in the digital world there is no decomposition, but the energy is gone. The words remain, unchanging. No or very few pages are re-introduced for a long time, months to years.</p>
<p>I find this very curious. Most of my waking life is spent interacting with computers. And most of that interacting is on the internet, consuming and commenting and feeding email, mailing lists, blogs, and myriad fora. How strange then, that my own piece of web real estate is so poorly maintained.</p>
<p>I don't know that this current renewal will, in fact, be a renewal, resulting in growth. I don't know that it won't either. I just know that this time around I'm somewhat more interested in watching the play of the impulses and energy behind the effort than the particulars of what system will be used this time around.</p>
<p><em>((which, if you're curious, will be <a href="http://posativ.org/acrylamid/">acrylamid</a>, though after reading about <a href="http://en.wikipedia.org/wiki/Acrylamide">acrylamide</a> I'm wondering about the choice in project name... ;-))</em></p>
<p>Okay, enough of the intro, on with the game!</p>Record internet radio with VLC, updatedtag:www.maphew.com,2013-01-16:/HowTo/record-internet-radio-with-vlc-updated2013-01-16T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>I needed to record a CBC radio piece today. For most cases, it appears
the easiest route is to use <em>streamtuner</em> and <em>streamripper</em>, however
there is something about my particular computer or CBC’s streaming style
that made it difficult for me. Since it took a couple of hours to figure
it all out, I figured I’d better save it for future reference. So here
is a vlc only method which works, but with more clicks and it’s is
easier to select a combination of options that don’t work than do. This
is the combination which finally worked for me:</p>
<ul>
<li>start VLC</li>
<li>File > Open Network Stream</li>
<li>for HTTP/HTTPS/FTP/MMS:
<a href="http://www.cbc.ca/listen/streams/r1_whitehorse_32.html" title="http://www.cbc.ca/listen/streams/r1_whitehorse_32.html">http://www.cbc.ca/listen/streams/r1_whitehorse_32.html</a></li>
<li>check ’steam/save’, select ‘Settings’, then ‘Outputs’:</li>
<li>check ‘Play Locally’</li>
<li>check ‘File’, set filename as you wish</li>
<li>Encapsulation method: ‘Raw’</li>
<li>check ‘Audio codec’, and select ‘mpga’ from the list, bitrate as you
like (96kb is okay for this particular stream)</li>
</ul>
<p>It’s the audio codec part which tripped me up the most, even though I
was saving mp3 selecting mp3 as a code yields a 0 byte file.</p>
<p>For command line use, copy the source and destination urls from the
dialogs created above, however it is necessary to change the syntax
somewhat. Namely the <code>:sout</code> option needs to be single quoted.</p>
<blockquote>
<p><code>vlc http://www.cbc.ca/listen/streams/r1_whitehorse_32.htm ‘:sout=#transcode{acodec=mpga,ab=96,channels=2}:duplicate{dst=display,dst=std{access=file,mux=raw,dst="/home/matt/Radio/cbc-north.mp3"}’</code></p>
</blockquote>
<p>Here is the same in a bash script, you should only need to edit the
INSTREAM and OUTFILE lines to use elsewhere</p>
<blockquote>
<p><code>#!/bin/sh NOW=$(date +%F_%k-%M) INSTREAM=http://www.cbc.ca/listen/streams/r1_...ehorse_32.html OUTFILE=/home/shared/Audio/Radio/cbc-north_$NOW.mp3 echo … echo …Recording CBC North internet radio to echo $OUTFILE echo …expect a 5 second delay before hearing anything echo …and a few “access_mms” errors prior to that. echo … /usr/bin/vlc $INSTREAM ‘:sout=#transcode{acodec=mpga,ab=96,channels=2}:duplicate{dst=display,dst=std{access=file,mux=raw,dst="'$OUTFILE'"}}’</code></p>
</blockquote>
<p><strong>2013-Jan-16</strong>\</p>
<p>Thanks for writing up the howto on the use of headless vlc to capture
streaming audio.\
\
It took a little experimentation to make it work on my hardware (an
arm-based pogoplug v4). I don't know if the changes are due to newer
versions of vlc or pertain more to how it was built for the pacman
database that arch linux uses (the os that's running on my pp4). Either
way, I thought I'd send what I did your way in case you were interested
in updating/adding to the howto.\</p>
<blockquote>
<p><code>#!/bin/sh NOW=$(date +%F_%k-%M) INSTREAM=http://www.wwoz.org/listen/lo OUTFILE=/home/user1/wwoz_$NOW. mp3 echo echo recording WWOZ echo cvlc $INSTREAM --sout "#transcode{acodec=mpga,ab=96,channels=2}:std{access=file,mux=raw,dst="$OUTFILE"}"</code>\</p>
</blockquote>
<p>I wound up using cvlc as opposed to vlc so that it wouldn't complain
about dbus access (lack of X11 really). There's something I have yet to
sort on the output file side as libmp3lame complains "[libmp3lame @
0xb3e80] channel_layout not specified". It does record, but what the
exact parameters it writes the file with I haven't yet looked into.\
\
Regards,\
John\
\
<em>\<John D. Doyle III></em></p>
<hr />
<p>Sources:</p>
<ul>
<li><a href="http://ubuntuforums.org/showpost.php?p=4447337" title="http://ubuntuforums.org/showpost.php?p=4447337">HOWTO: StreamTuner + StreamRipper, another way to get
music</a></li>
<li><a href="http://www.linux.com/articles/43721" title="http://www.linux.com/articles/43721">Building an Internet radio recorder with VLC
Player</a><a href="http://www.linux.com/articles/43721" title="http://www.linux.com/articles/43721">\
</a></li>
<li><a href="http://forum.videolan.org/viewtopic.php?f=4&t=40822" title="http://forum.videolan.org/viewtopic.php?f=4&t=40822">Command line HTTP/File stream: nothing to
play</a></li>
</ul>What does your Map projection say about you?tag:www.maphew.com,2011-11-14:/Other/what-does-your-map-projection-say-about-you2011-11-14T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p><a href="http://xkcd.com/">Xkcd</a> has a wonderful <a href="http://xkcd.com/977/">comic on map
projections</a> in common and
not so common use, and what each of them have to say about the
personalities of those who like them. A blog post and accompanying
conversation at <a href="http://www.explainxkcd.com/wiki/index.php?title=977:_Map_Projections">Explain
XKCD</a>
fills out some of the jokes.</p>
<p>I liked so much I've reworked it into a 10"x12" mini-poster for
printing:</p>
<p><a href="http://www.maphew.com/files/65/%3dxkcd_-_Map_Projections.pdf"><img alt="xkcd-map-projections_10x12in.png" src="http://www.maphew.com/files/66/%3dxkcd-map-projections_10x12in7902.png?size=webview"/></a></p>Wanted: dead simple argparse examplestag:www.maphew.com,2011-09-15:/Itches/wanted-dead-simple-argparse-examples2011-09-15T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>The standard
<a href="http://docs.python.org/library/argparse.html">documentation</a>
for the
<a href="http://www.doughellmann.com/PyMOTW/argparse/">argparse</a>
python module, while excellent I'm sure, is too much for my tiny brain
to grasp. I don't need to do math on the command line or meddle with
formatting lines on the screen or change option characters. All I want
to do is "<em>if arg is A, do this, if B do that, if none of the above show
help</em> <em>and quit</em>"</p>appytag:www.maphew.com,2011-09-01:/Projects/appy2011-09-01T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>I’ve been sporadically working on a little command line installer for
OSGeo4W, called
<a href="http://code.google.com/p/maphew/wiki/AptGetLike" title="http://code.google.com/p/maphew/wiki/AptGetLike">apt</a>.
It is based on
<a href="http://lilypond.org/~janneke/software/cyg-apt" title="http://lilypond.org/%7Ejanneke/software/cyg-apt">cyg-apt</a>,
a command line apt-get-like installer for cygwin byJan Nieuwenhuizen. I
currently have two forks. One for o4w alone, which I’ve renamed apt, and
the second for cygwin alone, which I’ve left as cyg-apt. Forking is not
a good idea when there is so much in common, so I’m going to try and
bring them together. One app, two installers. Neither name fits this new
job so I’m going to use appy for the time being. It will have to change
eventually as there’s another, much bigger, python project with the
<a href="http://appyframework.org/" title="http://appyframework.org/">appy name</a>, but
I don’t want to spend hours agonizing over a name. Again.
<img alt=":)" src="http://www.maphew.com\www.uttara.ca\blog\wp-includes\images\smilies\icon_smile.gif"/></p>
<p>The appy series of posts will chronicle my misadventures as I try and
teach myself programming by making this small tool a little bit more
useful. To me anyway!</p>
<p><a href="http://code.google.com/p/maphew/source/browse/trunk/other/cyg-apt.py" title="http://code.google.com/p/maphew/source/browse/trunk/other/cyg-apt.py">Matt’s
cyg-apt</a>:
my only claim to (in)famy here is converting system calls to md5sum,
tar, etc. to pure python. So now the only pre-requisite is python. All
the real work and intelligence in this program is a result of Jan’s
work.</p>
<p>I created a <a href="http://code.google.com/p/maphew/downloads/list" title="http://code.google.com/p/maphew/downloads/list">standalone
exe</a>
of the o4w version which can be used to <a href="http://code.google.com/p/maphew/wiki/OsgeoFromScratch" title="http://code.google.com/p/maphew/wiki/OsgeoFromScratch">install o4w from
scratch</a>,
but at 2.5mb it’s a little big for what it actually does.</p>
<hr />
<p>Things are going really well. I'm nothing short of astounded with what
me, a nonprogrammer, has been able to accomplish in a few dozen hours
(albeit spread over many evenings over a year). Why just last night in
cyg-apt I added a function to <a href="http://code.google.com/p/maphew/source/detail?spec=svn94&r=94" title="http://code.google.com/p/maphew/source/detail?spec=svn94&r=94">use the last used
mirror</a>
instead of hardcoding it. Concieved, thought about, and implemented all
in one session. Now <strong>that</strong> is cool!</p>
<p><em>(many moons later)</em> I love python, it makes me feel like I know
what I'm doing. ;-) Around two hours ago I discovered apt was broken
because the server <a href="http://trac.osgeo.org/osgeo4w/ticket/173" title="http://trac.osgeo.org/osgeo4w/ticket/173">changed it's ini
format</a>.
One hour later I had it fixed, tested and committed with a routine I've
never used before.</p>
<h5>2011-09-01 a major milestone reached, <a href="http://trac.osgeo.org/osgeo4w/wiki/pkg-apt" title="http://trac.osgeo.org/osgeo4w/wiki/pkg-apt">Apt</a> eats Apt!</h5>
<p>Last night I completed building <strong>a package of apt.exe can upgrade
itself</strong>. This means, I think, that now apt should be able to replace
any and all packages in Osgeo4W in one go including python. There is
much left to do, it's only been tested on two computers so far, and it
consumes way more diskspace and bandwidth than I think it should, but it
works!</p>
<p>To try it out, point your existing apt or o4w setup program (see
<em>Testing Repository</em>at end of <a href="http://trac.osgeo.org/osgeo4w/wiki/RequiresPython" title="http://trac.osgeo.org/osgeo4w/wiki/RequiresPython">this
page</a>)
at
<a href="http://download.osgeo.org/osgeo4w/testing/" title="http://download.osgeo.org/osgeo4w/testing/">http://download.osgeo.org/osgeo4w/testing/</a>
and upgrade away.</p>
<div class="highlight"><pre><span class="n">apt</span> <span class="o">--</span><span class="n">mirror</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">download</span><span class="p">.</span><span class="n">osgeo</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">osgeo4w</span><span class="o">/</span><span class="n">testing</span><span class="o">/</span> <span class="n">update</span>
<span class="n">apt</span> <span class="n">new</span>
<span class="n">apt</span> <span class="n">upgrade</span>
</pre></div>
<p>After this running apt with no parameters should yield the usual help
page with a version number at the top that resembles:</p>
<div class="highlight"><pre><span class="o">-</span><span class="p">={</span> <span class="n">Id</span><span class="p">:</span> <span class="n">apt</span><span class="p">.</span><span class="n">py</span> 1179 2011<span class="o">-</span>09<span class="o">-</span>01 12<span class="p">:</span>42<span class="p">:</span>24<span class="n">Z</span> <span class="n">maphew</span> <span class="p">}=</span><span class="o">-</span>
<span class="n">apt</span> <span class="p">[</span><span class="n">OPTION</span><span class="p">]...</span> <span class="n">COMMAND</span> <span class="p">[</span><span class="n">PACKAGE</span><span class="p">]...</span>
</pre></div>
<p>If the first line is just [option] etc. the new apt.exe hasn't been
activated. Running %osgeo4w_root%\bin\apt.bat should fix that. If not
please let me know.</p>Mindtouchtag:www.maphew.com,2011-08-03:/web-dev/mindtouch2011-08-03T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<h5>v10 license file for Core</h5>
<p><a href="http://developer.mindtouch.com/en" title="http://developer.mindtouch.com/en">Mindtouch</a>
now consolidates all of it's products into a single set of packages,
which is good, however this necessitates contacting HQ for a license
file even for the open source Core version. Some are bothered by this
and don't want to give Mindtouch Corp their email address. If you are in
this group feel free to use the <a href="http://www.maphew.com/files/64/%3dmindtouch-v10-core-license.xml" title="/@api/deki/files/64/=mindtouch-v10-core-license.xml">attached license
file</a>
(rename to 'license.xml'). It is "licensed" to
<a href="http://www.mailinator.com/maildir.jsp?email=mindtouch-core&x=0&y=0" title="http://www.mailinator.com/maildir.jsp?email=mindtouch-core&x=0&y=0">mindtouch-core@safetymail.info</a>.</p>
<h5>Downgrade from Enterprise to Core</h5>
<p>As a result of <a href="http://forums.developer.mindtouch.com/showthread.php?6902-Downgrading-Mindtouch-Standard-to-Mindtouch-Core" title="http://forums.developer.mindtouch.com/showthread.php?6902-Downgrading-Mindtouch-Standard-to-Mindtouch-Core">this
thread</a>
I periodically get private messages asking how to downgrade to the open
source. It's really very simple (though I don't know if this is still
true with v10):</p>
<blockquote>
<p>hi maphew, we are not allowed to talk about that in the forum...
don't know why! so, just delete the license.xml, restart dekihost and
you're back on core =)</p>
<p>best regards,<br />
---snip---</p>
</blockquote>Police make it hard to reporttag:www.maphew.com,2011-07-06:/Musings/police-make-it-hard-to-report2011-07-06T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>Police make it hard to report things to them.</p>
<p>Last week while on a canoe trip we spotted a minivan crashed in the
river, at the bottom of a 30m+ cliff. From the open driver's side door
It looked like it had been pushed over the edge, and, judging from the
multi-level water stains on the side, had been there for a week or two
at least. Still, being good citizens when we got to somewhere we could
make a call we phoned the police just in case nobody had reported it
yet. They received the report easily enough but then demanded name,
phone number, street address, a few other things I don't recall, and
birthdate. Needing to know name and number makes sense, address is a bit
more intrusive, but okay, who knows, maybe I'm a disgruntled lover
trying to get someone in trouble. But birthdate? Take a hike! That's
irrelevant to the issue at hand and none of their business.</p>
<p>This morning while walking the dog along a bush trail I found a bike
that looked stolen or abandoned. Late this evening it was still there.
Now I know from past experience that a bike left on it's own in the
trees will very likely soon end up a kilometer or two down the trail
smashed and bent, and with some of the better bits missing altogether.
An unknown faceless someone would likely be very grateful to have it
moved somewhere for safekeeping and reported to the authorities, meaning
the police, and eventually returned safely home to them. And I'd feel
pretty good for having played the good samaritan. Trouble is I don't
want the police treating me like a suspect when all I'm trying to do is
hand someone else a good turn.</p>
<p>I'll call them, eventually, an unknown faceless someone deserves it, but
I have to overcome resistance to do so. Maybe next time I see something
by the trail or in the water I'll just keep going. Come to think of it,
maybe that's why I have the past experience in the first place of seeing
unattended things transformed into junk and garbage. Others before me
had decided it's just not worth the hassle; "nothing to see here, move
along." That's sad.</p>Pale Blue Dottag:www.maphew.com,2011-05-17:/Musings/pale-blue-dot2011-05-17T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<div style="float:right;">
<img src="http://www.maphew.com/files/43/%3dPale_Blue_Dot_-_small7902.jpg" alt="Pale Blue Dot - small.jpg"/>
</div>
<p><a href="http://www.ijourney.org/index.php?tid=682" title="http://www.ijourney.org/index.php?tid=682">This article</a>
came across my desk and inspired me to make a poster. Well, it inspired
me to find a poster, but I didn't locate one that resonated just the
right way with me, so I made my own.</p>
<p><em>A portion of the first ever 'portrait' of the solar system taken by
Voyager 1 in 1990 from a distance of more than 4 billion miles from
Earth. Our world is a mere point of light, a crescent only 0.12 pixels
in size, coincidentally right in the center of a scattered light ray,
resulting from taking the image so close to the sun.</p>
<p>"We succeeded in taking that picture , and, if you look at it, you see
a dot. That's here. That's home. That's us. On it, everyone you ever
heard of, every human being who ever lived, lived out their lives. The
aggregate of all our joys and sufferings, thousands of confident
religions, ideologies and economic doctrines, every hunter and forager,
every hero and coward, every creator and destroyer of civilizations,
every king and peasant, every young couple in love, every hopeful child,
every mother and father, every inventor and explorer, every teacher of
morals, every corrupt politician, every superstar, every supreme leader,
every saint and sinner in the history of our species, lived there on a
mote of dust, suspended in a sunbeam.</p>
<p>The earth is a very small stage in a vast cosmic arena. Think of the
rivers of blood spilled by all those generals and emperors so that in
glory and in triumph they could become the momentary masters of a
fraction of a dot. Think of the endless cruelties visited by the
inhabitants of one corner of the dot on scarcely distinguishable
inhabitants of some other corner of the dot. How frequent their
misunderstandings, how eager they are to kill one another, how fervent
their hatreds. Our posturings, our imagined self-importance, the
delusion that we have some privileged position in the universe, are
challenged by this point of pale light.</p>
<p>Our planet is a lonely speck in the great enveloping cosmic dark. In
our obscurity -- in all this vastness -- there is no hint that help will
come from elsewhere to save us from ourselves. It is up to us. It's been
said that astronomy is a humbling, and I might add, a character-building
experience. To my mind, there is perhaps no better demonstration of the
folly of human conceits than this distant image of our tiny world. To
me, it underscores our responsibility to deal more kindly and
compassionately with one another and to preserve and cherish that pale
blue dot, the only home we've ever known.</p>
<p>-- Carl Sagan, "Reflections on a Mote of Dust"
</em></p>
<p><a href="http://www.howardism.org/Other/Pale_Blue_Dot.html"><strong>Listen</strong> to Carl read</a> this passage<br />
<a href="http://bit.ly/pale-blue-dot-video"><strong>Watch</strong> the speech</a> set to music and imagery*<br />
<a href="http://www.zazzle.ca/pale_blue_dot_print-228025123154760874"><strong>Purchase</strong> a hardcopy</a> in a varity of sizes, paper & framing options<br />
<a href="http://www.maphew.com/files/46/%3dPale_Blue_Dot.zip"><strong>Download</strong></a> in editable svg format (350k) </p>
<p><a href="http://flattr.com/thing/276966/Pale-Blue-Dot"><img alt="Flattr this" src="http://api.flattr.com/button/flattr-badge-large.png" title="Flattr this"/></a></p>
<h5>Sources & Inspiration:</h5>
<ul>
<li>iJourney -
<a href="http://www.ijourney.org/index.php?tid=682">http://www.ijourney.org/index.php?tid=682</a></li>
<li>*Carl Sagan "Pale Blue Dot" (New Version) Spread This! -
<a href="http://www.youtube.com/watch?v=luAteAz3WQ0">http://www.youtube.com/watch?v=luAteAz3WQ0</a></li>
<li>Sky Image Lab -
<a href="http://www.skyimagelab.com/pale-blue-dot.html">http://www.skyimagelab.com/pale-blue-dot.html</a></li>
<li>Matthew Beckler -
<a href="http://www.mbeckler.org/perspective/">http://www.mbeckler.org/perspective/</a></li>
<li>Howard Abrams -
<a href="http://www.howardism.org/Other/Pale_Blue_Dot.html">http://www.howardism.org/Other/Pale_Blue_Dot.html</a> (Howard has an audio only version; took quite awhile to find.)</li>
</ul>
<p><a href="http://i.creativecommons.org/l/by-sa/2.5/ca/88x31.png" title="Creative Commons License"><img alt="Creative Commons License" src="http://i.creativecommons.org/l/by-sa/2.5/ca/88x31.png"/></a></p>
<p>Pale Blue Dot by <a href="http://www.maphew.com\Musings\Pale_Blue_Dot.html">Matt Wilkie</a> is licensed under
a <a href="http://creativecommons.org/licenses/by-sa/2.5/ca/">Creative Commons Attribution-Share Alike 2.5 Canada
License</a>
Based on a work at
<a href="http://www.skyimagelab.com/pale-blue-dot.html">www.skyimagelab.com</a>.
Permissions beyond the scope of this license may be available at
<a href="http://www.maphew.com\Musings\pale-blue-dot\mailto:maphew@gmail.com">maphew@gmail.com</a>.</p>Time Traveller Drop Pointstag:www.maphew.com,2011-05-17:/Other/time-traveller-drop-points2011-05-17T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>I was cleaning out old email and ran across this important message which
fell through the cracks. I don't have the time to fulfill this urgent
request, but thought perhaps I could help by getting the word out to
someone who can. Thus was born the <a href="http://maps.google.ca/maps/ms?hl=en&ie=UTF8&t=p&msa=0&msid=110062113008218362869.00047dc44105664577020&ll=42.494251,-71.142998&spn=0.011075,0.017252&z=14&source=embed" title="http://maps.google.ca/maps/ms?hl=en&ie=UTF8&t=p&msa=0&msid=110062113008218362869.00047dc44105664577020&ll=42.494251,-71.142998&spn=0.011075,0.017252&z=14&source=embed">Time Traveller drop
points</a>:</p>
<blockquote>
<p>A web map service for Time Travellers who need to arrange for pick up
and or delivery of specialty goods. On the go and need a supply dump
provisioned? Put your drop point and request here.\
\
<strong>NOTE</strong>: payment and delivery term arrangements are <strong>*strictly*</strong>
between traveller and deliverer. We don't want to know about it and
will not refund a deal gone bad.</p>
</blockquote>
<p><a href="http://flattr.com/thing/276987/Time-Traveller-Drop-Points" title="http://flattr.com/thing/276987/Time-Traveller-Drop-Points"><img alt="Flattr
this" src="http://www.maphew.com\api.flattr.com\button\flattr-badge-large.png" title="Flattr this"/></a></p>
<p>—–Forwarded Message—–\
Subject: Dimensional warp Generator Needed ubamz\
Date: 09 Aug 2003 16:33:10 +0000</p>
<p>Hello,</p>
<p>I’m a time traveler stuck here in 2003. Upon arriving here my
dimensional warp generator stopped working. I trusted a company here by
the name of LLC Lasers to repair my Generation 3 52 4350A watch unit,
and they fled on me. I am going to need a new DWG unit, prefereably the
rechargeable AMD wrist watch model with the GRC79 induction motor, four
I80200 warp stabilizers, 512GB of SRAM and the menu driven GUI with
front panel XID display.</p>
<p>I will take whatever model you have in stock, as long as its received
certification for being safe on carbon based life forms.</p>
<p>In terms of payment:</p>
<p>Payment can be made in Galactic Credits, Platinum gold, or 2003 currency
upon safe delivery of unit.</p>
<p>INSTRUCTIONS MUST BE FOLLOWED EXACTLY:</p>
<p>Please transport unit in either a brown paper bag or box to below
coordinates on Saturday August 9th at (exactly 3:00pm) Eastern Standard
Time on the dot. A few minutes prior will be ok, but it cannot be after.
If you miss this timeframe please email me. I will not be there prior to
2:45pm EST, (so do not transport before then).</p>
<p>\
View <a href="http://maps.google.ca/maps/ms?hl=en&ie=UTF8&t=p&msa=0&msid=110062113008218362869.00047dc44105664577020&ll=42.494251,-71.142998&spn=0.011075,0.017252&z=14&source=embed" title="http://maps.google.ca/maps/ms?hl=en&ie=UTF8&t=p&msa=0&msid=110062113008218362869.00047dc44105664577020&ll=42.494251,-71.142998&spn=0.011075,0.017252&z=14&source=embed">Time Traveller drop
points</a>
in a larger map</p>
<p>Item is to be delivered at Majority Lane in Woburn, Massachusetts
located at: Latitude N 42.49430 & Longitude W 071.14275 and the
Elevation is 140 feet.</p>
<p>WARNING: DO NOT ATTEMPT TO TRANSPORT ITEM BY REGULAR MEANS OF
TELEPORTATION. THEY ARE MONITORING AND WILL REDIRECT THE SIGNAL!!</p>
<p>I DO NOT CARE HOW YOU HAVE TO GET IT HERE, JUST DO IT IN A WAY THAT NO
SPYING EYES WILL POSSIBLY BE ABLE TO REDIRECT or BLOCK THE TRANSFERENCE.
IT IS VERY IMPORTANT THAT YOU BE ABLE TO MONITOR THE TRANSFER.</p>
<p>HOW ARE YOU GOING TO SEND IT SO THAT THEY CANNOT REDIRECT OR BLOCK IT???
If in doubt do not transport actual unit until your method of transfer
can be confirmed as a success. You just might need to send a
intergalactic courier to deliver item safely to me. If so be VERY
careful at how they approach me.</p>
<p>It is best if you send a intergalactic courier to deliver, this way you
can be certain the unit arrives ok, However If you are certain that you
have the means to teleport unit in a safe manner please send a
(separate) email to me at:
<a href="http://www.maphew.com\Other\time-traveller-drop-points\mailto:webmaster@federalfundingprogram.com" title="mailto:webmaster@federalfundingprogram.com">webmaster@federalfundingprogram.com</a>
only after unit has been safely delivered with payment instructions.</p>
<p>Do not reply directly back to this email as it will only be bounced back
to you.</p>Decollaring with nearblacktag:www.maphew.com,2011-04-28:/Itches/decollaring-with-nearblack2011-04-28T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p><em><a href="http://www.maphew.com/files/62/%3dto-be-decollared.jpg" title="to-be-decollared.jpg"><img alt="to-be-decollared.jpg" src="http://www.maphew.com/files/62/%3dto-be-decollared7902.jpg?size=webview"/></a>The
<a href="http://www.maphew.com/files/63/%3dto-be-decollared.tif" title="/@api/deki/files/63/=to-be-decollared.tif">attached
geotiff</a>
image has two collars to be removed. The outermost consists of pure
black (0,0,0) while the innermost is off-white (ranges from 240 thru
255). I used
<a href="http://code.google.com/p/maphew/source/browse/trunk/gis/o4w_extras/bin/gdalsetnull.py" title="http://code.google.com/p/maphew/source/browse/trunk/gis/o4w_extras/bin/gdalsetnull.py">gdalsetnull</a>
to change 0,0,0 to nodata, then ran
`<a href="http://www.gdal.org/nearblack.html" title="http://www.gdal.org/nearblack.html">nearblack</a>
-white` on the result. I detect no difference in the output image even
with a very large fuzz factor (-near 50) .</em></p>
<p><em>What am I doing wrong?</em></p>
<h5>Answers</h5>
<p>From Even Rouault I learned that <a href="http://osgeo-org.1803224.n2.nabble.com/gdal-dev-nearblack-troubles-tp6310029p6310401.html" title="http://osgeo-org.1803224.n2.nabble.com/gdal-dev-nearblack-troubles-tp6310029p6310401.html">nearblack doesn't grok
nodata</a>,
it just looks at the pixel values. Consequently it never sees the white
collar, except a small portion at the very top of the image.</p>
<p>Thankfully Luke Pinner, who's come to my aid
<a href="http://www.maphew.com\Itches\Projects\appy.html" title="Projects/appy">before</a>, has an elegant solution
using a VRT intermediate file
<a href="http://osgeo-org.1803224.n2.nabble.com/gdal-dev-nearblack-troubles-tp6310029p6311500.html" title="http://osgeo-org.1803224.n2.nabble.com/gdal-dev-nearblack-troubles-tp6310029p6311500.html">(ref)</a>:</p>
<div class="highlight"><pre><span class="c">#Change black to white</span>
<span class="n">gdalbuildvrt</span> <span class="o">-</span><span class="n">srcnodata</span> 0 <span class="o">-</span><span class="n">vrtnodata</span> 255 <span class="n">to</span><span class="o">-</span><span class="n">be</span><span class="o">-</span><span class="n">decollared</span><span class="p">.</span><span class="n">vrt</span>
<span class="n">to</span><span class="o">-</span><span class="n">be</span><span class="o">-</span><span class="n">decollared</span><span class="p">.</span><span class="n">tif</span>
<span class="c">#Run nearblack -white</span>
<span class="n">nearblack</span> <span class="o">-</span><span class="nb">white</span> <span class="o">-</span><span class="n">o</span> <span class="n">decollared</span><span class="p">.</span><span class="n">tif</span> <span class="n">to</span><span class="o">-</span><span class="n">be</span><span class="o">-</span><span class="n">decollared</span><span class="p">.</span><span class="n">vrt</span>
</pre></div>ipy and arcgisscriptingtag:www.maphew.com,2011-04-21:/Projects/ipy-and-arcgisscripting2011-04-21T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>….some further experiments with
<a href="http://ipython.scipy.org/" title="http://ipython.scipy.org/">ipython</a>and
<a href="http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?id=709&pid=706&topicname=Writing_Python_scripts" title="http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?id=709&pid=706&topicname=Writing_Python_scripts">ArcGIS
python</a>
geoprocessing. Here is a simple script to convert a bunch of coverages
to shapefile. There are 9 coverages occupying 3mb, it takes about 6
minutes, consumes 2 processors to 80-90% capacity and chews through
300mb of ram:</p>
<blockquote>
<div class="highlight"><pre><span class="n">cd</span> <span class="n">w</span><span class="p">:</span><span class="o">/</span><span class="n">Env</span><span class="o">-</span><span class="n">dat</span><span class="p">.</span>003<span class="o">/</span>2007<span class="o">-</span><span class="n">March</span><span class="o">/</span><span class="n">workspace</span><span class="o">/</span><span class="n">envy_ed2import</span> <span class="n">arcgisscripting</span>
<span class="n">gp</span> <span class="p">=</span> <span class="n">arcgisscripting</span><span class="p">.</span><span class="n">create</span><span class="p">()</span>
<span class="n">gp</span><span class="p">.</span><span class="n">Workspace</span> <span class="p">=</span> ‘<span class="o">./</span>’
<span class="n">todo</span> <span class="p">=</span> !<span class="n">dir</span> <span class="o">/</span><span class="n">b</span> <span class="n">fwtc</span><span class="o">*</span>
<span class="k">for</span> <span class="n">cov</span> <span class="n">in</span> <span class="n">todo</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">print</span> <span class="n">cov</span>
<span class="n">gp</span><span class="p">.</span><span class="n">FeatureClassToShapefile</span><span class="p">(</span> <span class="n">cov</span> <span class="o">+</span> ‘<span class="o">/</span><span class="n">arc</span>’<span class="p">,</span> ‘<span class="o">./</span><span class="n">shp</span>’<span class="p">)</span>
<span class="n">except</span><span class="p">:</span>
<span class="n">print</span> <span class="n">gp</span><span class="p">.</span><span class="n">GetMessages</span><span class="p">()</span>
</pre></div>
</blockquote>
<p>Contrast that to using
<a href="http://www.fwtools.org/" title="http://www.fwtools.org/">fwtools</a>
<a href="http://www.gdal.org/ogr/ogr2ogr.html" title="http://www.gdal.org/ogr/ogr2ogr.html">ogr2ogr</a>,
which takes 10 seconds:</p>
<blockquote>
<div class="highlight"><pre><span class="k">for</span> <span class="c">%a in (fwtc*) do ogr2ogr -f “esri shapefile” ogr\%a %a</span>
</pre></div>
</blockquote>
<p>The problem is, ogr2ogr doesn’t currently translate everything arcpy
does. It leaves behind the metadata and fills voids (though these can be
filtered out later as the attributes are empty, but that is an extra
step). There’s also no way (that I can see) to select only arcs or only
points.</p>
<h5>(June 6th) another episode in the ogr vs arcgis battle:</h5>
<p>Goal: <strong><em>Project 70 shapefiles in various subdirectories occupying
300mb</em></strong>.</p>
<h6><strong>ArcGIS BatchProject interactively:</strong></h6>
<ul>
<li>search catalog for k:\incoming\NHN*.shp</li>
<li>fire up Batch Project toolbox</li>
<li>drag’n'drop search results (wait 2+ minutes for verification step to
complete)</li>
<li>run</li>
<li>oops, error in selected projection parameters. failed.</li>
<li>try again, this time using Command Line Window to see what’s going
on. (2+ minutes for verification)</li>
<li>figure out problem, fix parameters (2+ min for verification, which
repeats every time you attempt to Insert Variable or other command
change)</li>
<li>execute the command. (Finished in 21 minutes, 43 seconds.)</li>
</ul>
<p><strong>total arc time: >45 minutes.</strong></p>
<h6><strong>ogr2ogr:</strong></h6>
<ul>
<li>research command syntax, examine old notes and scripts (5 minutes)</li>
<li>build and test command line (2 minutes)</li>
<li>execute (7 minutes, 12 seconds)</li>
</ul>
<p><strong>total ogr time: \~15 minutes.</strong>So even if we subtract the debugging
time for the problematic projection parameters (which have never caused
me trouble before, but that is another story), ogr is still twice as
fast. Which could be 3 times as fast if the command syntax were
internalized and I didn’t have to look it up. Guess where I’m going to
be spending some optimisation energy!</p>
<p>And now for the initial impetus for posting this comment, saving my
command line!</p>
<blockquote>
<div class="highlight"><pre><span class="k">for</span> <span class="o">/</span><span class="n">r</span> <span class="c">%a in (nhn*.shp) do @ogr2ogr ogr %a -t_srs z:\Arcutils\prj\YkAlbers.proj4</span>
</pre></div>
</blockquote>Don't show internal boundaries, or: Arcinfo Regions, we miss you!tag:www.maphew.com,2011-04-18:/Itches/don-t-show-internal-boundaries-or-arcinfo-regions-we-miss-you2011-04-18T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>this page is to support the <a href="http://gis.stackexchange.com/questions/8571/dont-show-internal-boundaries-or-arcinfo-regions-we-miss-you">GIS.stackexhange.com
question</a>
of the same name, namely how does one most efficiently use arcgis and
feature classes to achieve some the same things which used to be
possible with arcinfo workstation and region sub-coverages.</p>
<p><img alt="" src="http://www.maphew.com\i.imgur.com\8NpAT.gif"/></p>
<p>Please see attached archives for a v10 <a href="http://www.maphew.com/files/60/%3dLake-cartography-wanted.zip" title="/@api/deki/files/60/=Lake-cartography-wanted.zip">map
package</a>
which demonstrates the problem and (simulated) desired results, and the
<a href="http://www.maphew.com/files/61/%3dwatercov.zip">source arcinfo
coverage</a>
from which the simulation is built.</p>COMfounded pythontag:www.maphew.com,2011-01-25:/Projects/Canvec/comfounded-python2011-01-25T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>I think my python-COM environment is messed up.</p>
<p><strong>Got</strong></p>
<div class="highlight"><pre><span class="n">D</span><span class="p">:</span><span class="o">\></span> <span class="n">python</span> <span class="n">list</span><span class="o">-</span><span class="n">fc</span><span class="p">.</span><span class="n">py</span>
<span class="p">.</span><span class="o">./</span><span class="n">Scratch</span><span class="o">/</span><span class="n">blank_canvec</span><span class="p">.</span><span class="n">gdb</span>
<span class="o"><</span><span class="n">COMObject</span> <span class="o"><</span><span class="n">unknown</span><span class="o">>></span>
</pre></div>
<p>and <strong>expected</strong>something <a href="http://bit.ly/cfvwxY" title="http://bit.ly/cfvwxY">like
this</a>:</p>
<p>../Scratch/blank_canvec.gdb\
[u'counties.shp', u'roads.shp']\
counties.shp\
roads.shp</p>
<h6>Possible culprits:</h6>
<p><img alt="" src="http://www.maphew.com/files/56/%3dstatic-python-dlls.png"/></p>
<ul>
<li>using Mark Cederholm's
<a href="http://gis.stackexchange.com/questions/80/how-do-i-access-arcobjects-from-python" title="http://gis.stackexchange.com/questions/80/how-do-i-access-arcobjects-from-python">python+arcobjects</a>
recipe requires registering com stuff, which conflicts wth
PythonWin's com stuff.</li>
<li>I'm using <a href="http://www.yukongis.ca/How_To/Upgrading_ArcGIS_Python" title="http://www.yukongis.ca/How_To/Upgrading_ArcGIS_Python">Arcgis with python
2.6</a>
instead of the shipped 2.5. This hasn't proved problematic before,
but maybe I just haven't wondered into this section of the forest
before (which means I need to <a href="http://www.maphew.com\Projects\Canvec\Search_PATH_for_duplicates.html" title="Projects/Search PATH for duplicates">eat crow for Jason
Scheirer</a>).</li>
<li>I have concurrent python 2.5 and python 2.6 installs, 32 and 64bit,
and they are walking over each other</li>
<li>one of the many other programs which distribute static python dll's
is in path and colliding</li>
<li>I'm just very poor snake charmer. :)</li>
</ul>
<h5>Troubleshooting</h5>
<p>Win32 bug#3076741, <a href="https://sourceforge.net/tracker/?func=detail&aid=3076741&group_id=78018&atid=551954" title="https://sourceforge.net/tracker/?func=detail&aid=3076741&group_id=78018&atid=551954">"This object does not support
enumeration"</a>.
Using win32com.client.gencache registers com objects on the fly, fixes
one problem and creates others.</p>
<h6>update, 2011 Jan 25</h6>
<p>I think I've narrowed it to the win32 extension, as presented by the
<a href="http://www.yukongis.ca/How_To/Upgrading_ArcGIS_Python" title="http://www.yukongis.ca/How_To/Upgrading_ArcGIS_Python">arcgisscripting</a>wrapper.</p>
<p>I uninstalled all versions of python and cleared all directories from
PATH that had any kind of python??.dll, then installed python 2.5.4, and
numpy from the arcgis dvd (as per <a href="http://blogs.esri.com/Dev/blogs/geoprocessing/archive/2009/05/12/Using-Python-2.5.4-with-ArcGIS-9.3.1.aspx" title="http://blogs.esri.com/Dev/blogs/geoprocessing/archive/2009/05/12/Using-Python-2.5.4-with-ArcGIS-9.3.1.aspx">instructions
here</a>).
If <em>PYTHONPATH</em> contains ...\ArcGIS\bin, then we get the expected
results, if not, and we add arcgisscripting.py to Lib\site-packages:
win32com not found. If we then install the win32com package we get the
\<COMObject \<unknown>> error.</p>
<p>The readme for PythonWin on the install dvd says <em>"ArcGIS no longer
installs PythonWin because the recommended methodology for creating the
geoprocessor has changed to using a new Python module that ESRI has
developed ...".</em></p>
<p>Sooooo, arcgisscripting.py and win32com work for most things, but not
for gp.ListFields() or gp.ListFeatureClasses() functions. I bet this
also applies to other list functions too, see <a href="http://resources.arcgis.com/content/kbase?fa=articleShow&d=34873" title="http://resources.arcgis.com/content/kbase?fa=articleShow&d=34873">changes in list
methods</a>
knowledgebase article. This means that python 2.6 and 2.7 can be used
with ArcGIS 9.x, but only if you stay away from the list functions.</p>Retrieve members of AD grouptag:www.maphew.com,2011-01-18:/HowTo/retrieve-members-of-ad-group2011-01-18T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p><em>How to retrieve members and email of an Active Directory group in a re-usable format.</em></p>
<p>Locate
<a href="http://technet.microsoft.com/en-us/library/cc755162(WS.10).aspx" title="http://technet.microsoft.com/en-us/library/cc755162(WS.10).aspx">dsget</a>
command line utility and put it in path.</p>
<p>Open an administrative command shell and:</p>
<div class="highlight"><pre><span class="n">dsget</span> <span class="n">group</span> "<span class="n">CN</span><span class="p">=</span><span class="n">mygroup</span><span class="p">,</span><span class="n">OU</span><span class="p">=</span><span class="n">mydepartment</span><span class="p">,</span><span class="n">OU</span><span class="p">=</span><span class="n">Users</span> <span class="n">and</span> <span class="n">Groups</span><span class="p">,</span><span class="n">DC</span><span class="p">=</span><span class="n">mydomain</span><span class="p">,</span><span class="n">DC</span><span class="p">=</span><span class="n">ca</span>" <span class="o">-</span><span class="n">members</span> <span class="o">|</span><span class="n">dsget</span> <span class="n">user</span> <span class="o">-</span><span class="n">samid</span> <span class="o">-</span><span class="n">display</span> <span class="o">-</span><span class="n">email</span>
</pre></div>
<p><img alt="dsget-list-name-emails.png" src="http://www.maphew.com\HowTo\retrieve-members-of-ad-group\"\files\59\=dsget-list-name-emails.png"/></p>Searching Active Directorytag:www.maphew.com,2011-01-18:/HowTo/searching-active-directory2011-01-18T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>This is from the <em>I can't believe I didn't know that and <strong>why</strong>haven't
I been using it for the last ten years!??</em> department.</p>
<p>Many times in the last decade I've<strong>needed to<strong><em>*search Active
Directory</em></strong>*for a person</strong>'s name, login name, what members belong to
what group and so on. Each time the need has been pressing enough I've
gone out and searched the 'net for solutions, and each time after a time
come up with a solution. Said time might have been 10 mintues or 2 hours
depending on what in particular I was trying to do. The solutions I've
found and used ranged from the easy to use
<a href="http://technet.microsoft.com/en-us/sysinternals/bb963907" title="http://technet.microsoft.com/en-us/sysinternals/bb963907">AD-Explorer</a>
to roll your own python or batchfile scripts (that have taken further
hours to develop to suit local circumstances).</p>
<p>Encountering the need again today, and realizing my local stash of
recipes was much <a href="http://www.maphew.com\HowTo\searching-active-directory\Retrieve_members_of_AD_group.html" title="How To/Retrieve members of AD group">too scary or
technical</a>
to hand off to our regular staff who would actually be using it, I
embarked on the quest again. Lo and behold, every version of Windows
from (at least) XP on up includes the built in <em>Find Users, Contacts,
and Groups</em>. It's buried under 'My Network Neighbourhood > Common
Tasks' which I'd given up on long ago as being a useless "my computer"
place to visit (when I want to see the network it's far more common to
want to see the workstations and servers near me and turned on than the
last 30 or more places I've been to). Who knew? (obviously not me!)</p>
<p>Anyway, here's the meat of it and you can't put it anywhere you want, so
it's actually convenient: Create a shortcut to <strong>rundll32.exe dsquery,
OpenQueryWindow</strong>. And that's it, you're done.</p>
<p>Sheesh.</p>
<p>Thank you Information Systems & Technology at University of Waterloo for
<a href="http://ist.uwaterloo.ca/security/howto/2006-12-13/" title="http://ist.uwaterloo.ca/security/howto/2006-12-13/">illuminating
me</a>.</p>Long Live the Aurora Text Editortag:www.maphew.com,2010-10-20:/Musings/long-live-the-aurora-text-editor2010-10-20T07:00:00Zmatt wilkiehttp://www.maphew.com/maphew+website@gmail.com<p>Thanks to the persistence of Kent Nassen and the generosity of Jeff
Wunderlich, the <a href="http://www-personal.umich.edu/~knassen/aurora.html" title="http://www-personal.umich.edu/%7Eknassen/aurora.html">Aurora Text
Editor</a>
can be downloaded and registered, “…the .bat file command I use to
register is: ac -reg “Jeff Wunderlich” “xujcwpkgzngqmqn” 1 Feel free to
share it with anyone on the net.”</p>
<p>So what’s the big deal? Well I covered most of it a few years ago in
<a href="http://web.archive.org/web/20080524194444/http://www.uttara.ca/patawi/matt/musings/favedit.html" title="http://web.archive.org/web/20080524194444/http://www.uttara.ca/patawi/matt/musings/favedit.html">Musing on the Favourite Text
Editor</a>
(slow link, involves time travel), but at the moment my biggest cause
for joy is that I can, once again, select a column and inserting a
character or range of characters (useful for turning fixed width text
records into delimited text), and filling a columnar selection with an
increasing pattern (1,2,3… a,b,c…) similar to Excel’s extend series
command.</p>
<p>Since I wrote the musing’s essay I’ve been a happy
<a href="http://cream.sourceforge.net/" title="http://cream.sourceforge.net/">Cream</a>
user,with occasional forays into
<a href="http://notepad-plus.sourceforge.net/" title="http://notepad-plus.sourceforge.net/">Notepad++</a>
when speed and portability (happy on a usb key) is desired. Now that
Aurora is around and permanently free I’m positively cheery. Yes I know
<a href="http://www.textpad.com/" title="http://www.textpad.com/">TextPad</a> can do
column fills, and TP is a very fine editor indeed. I bought it for work
after all, but it’s not cross platform and I don’t want to pay for it on
for every computer I use. Besides, Aurora is only a 650kb download.
Where else can you get so much editor bang for your buck? Seriously,
where else?</p>