links.lmorchard.com

#data

74 bookmarks tagged with #data · RSS Feed · ← All Tags

XML is a cheap DSL

https://unplannedobsolescence.com/blog/xml-cheap-dsl/
The lesson I took from this is that a universal data representation is worth its weight in gold. There are exactly two options in this category. In most cases you should choose JSON. If you need a DSL though, XML is by far the cheapest one, and the cost-efficiency of building on it will empower your team to spend their innovation budget elsewhere.
#data #xml #xpath
archive.org #38408

Backing up Spotify - Anna’s Blog

https://annas-archive.li/blog/backing-up-spotify.html
We backed up Spotify (metadata and music files). It’s distributed in bulk torrents (~300TB), grouped by popularity. This release includes the largest publicly available music metadata database with 256 million tracks and 186 million unique ISRCs. It’s the world’s first “preservation archive” for music which is fully open (meaning it can easily be mirrored by anyone with enough disk space), with 86 million music files, representing around 99.6% of listens.
#archives #data #metadata #music
archive.org #38139

Of the web

https://werd.io/of-the-web/
most web apps are on the web, but not of it. This seems right to me, and I suspect web instigators like Tim Berners-Lee would agree. The odd thing is that, for a while, desktop software did exactly this. Word would save a document in a particular format, and another application written by someone else entirely could open that document. When we moved to the cloud, we signed up for a computing environment that was as much about siloing and isolation as much as it was about ease of access and maintenance.
#apps #data #web #webdev
archive.org #38069

Why Tim Berners-Lee still believes in the web | The Verge

https://www.theverge.com/podcast/814552/tim-berners-lee-world-wide-web-ai-future-interview
The right place to end up is where you’re totally in control of your data. As the power of local devices becomes greater, then the amount of stuff which gets stored locally increases… a lot of the things that people are building out there on the web are local-first, meaning that they’re designed to be able to work without the cloud. They store data either on the web, or they store the data on your local device. The good place to be is that the data is all stored on your local device. The inference is happening in your device, and you’re in control.
#ai #data #privacy #webdev
archive.org #37942

xvw.lol - Indexing coffee with Notion

https://xvw.lol/en/articles/notion-coffee.html
To sum up, this article will present how I set up an infrastructure to index the coffees I taste, with the goal of providing precise metrics to help characterize my preferences, using the Notion tool, while also sharing some techniques and tricks I learned during the creation of this system.
#coffee #data #notion
archive.org #37896

Scripting News: Think different about developers

http://scripting.com/2025/10/23/133525.html#a133630
This is the same problem web devs have, we have to become resellers for Amazon S3. Why can't Amazon, who already has an account for every freaking person in the world, let the user own their own data, which I believe they would reallllly like. I don't want access to it, I just want to make great tools for them to use. And they're free to use other products with the data. This helps encourage data format standards. It's why the PC software market was so much more vibrant than the web developer market.
#amazons3 #data #selfhosting #webdev
archive.org #37852

Personal data storage is an idea whose time has come

https://blog.muni.town/personal-data-storage-idea/
Personal Data Storage has long since escaped containment as a concept pertaining to any specific protocol. Some implementations of it will be more mainstream than others, but pragmatic data coops can be protocol-agnostic and storage formats are transmutable. As long as we have sufficient control of our own data there will always be a way to restart our social graph and digital presence elsewhere in the event of platform collapse. Let’s make the web personal again.
#atprotocol #data #personal #solid #web #webdev
archive.org #37788

File over app — Steph Ango

https://stephango.com/file-over-app
File over app is a philosophy: if you want to create digital artifacts that last, they must be files you can control, in formats that are easy to retrieve and read. Use tools that give you this freedom. File over app is an appeal to tool makers: accept that all software is ephemeral, and give people ownership over their data.
#apps #data #fileoverapp #freedom #localfirst #tools
archive.org #37751

Synthetic Survey Data? It's Not Data

https://quantuxblog.com/synthetic-survey-data-its-not-data
My personal conclusion — as a matter of logic, empirical findings, statistical reasoning, and scientific principles — is that synthetic data has no place in survey research. I also believe the purported use cases for synthetic data are unconvincing; alternative approaches are superior in their results and are less risky in practice. And there are important questions about research ethics that each of us should consider.
#ai #data #genai #llms #research #science #surveys
archive.org #29413

xan/docs/LOVE_LETTER.md at master · medialab/xan

https://github.com/medialab/xan/blob/master/docs/LOVE_LETTER.md
It is therefore my intention, through this article, to write a love letter to this data format, often criticized for the wrong reasons, even more so when it is somehow deemed "cool" to hate on it. My point is not, far from it, to say that CSV is a silver bullet but rather to shine a light on some of the format's sometimes overlooked strengths.
#csv #data #formats
archive.org #29602

Maintaining control of our data with personal databases | N=1 (marcua’s blog)

https://blog.marcua.net/2025/01/05/maintaining-control-of-our-data-with-personal-databases
The fundamental problem is that in using most modern consumer-oriented applications, we lose control of the data we share with those applications. In creating documents, writing emails, or tracking our exercise or other activities, we share data with an application that’s useful for a particular purpose, but also allow the data live in a database controlled by the application’s owner. With the data outside of our control, it can then be restricted, lost, leaked, sold, resold, and exploited. The groups that control these databases accrue most of the benefits, and we accrue most of the costs.
#data #indieweb #personal #privacy #sql
archive.org #29727

Proposed standards for the care and feeding of user generated content – Jeffrey Zeldman Presents The Daily Report

http://www.zeldman.com/2012/07/25/proposed-standards-for-the-care-and-feeding-of-user-generated-content/
Treat our data like it matters. Keep it secure and protect our privacy, of course—but also maintain serious backups and respect our choice to delete any information we’ve contributed. No upload without download. Build in export capabilities from day one. If you close a system, support data rescue. Provide one financial quarter’s notice between announcing the shutdown and destroying any user-contributed content, public or private, and offer data export during this period. And beyond that three months? Make user-contributed content available for media-cost purchase for one year after shutdown.
#cloud #data #imported:pinboard #services #to:fb
archive.org #32714

Proposed standards for the care and feeding of user generated content – Jeffrey Zeldman Presents The Daily Report

http://www.zeldman.com/2012/07/25/proposed-standards-for-the-care-and-feeding-of-user-generated-content
Treat our data like it matters. Keep it secure and protect our privacy, of course—but also maintain serious backups and respect our choice to delete any information we’ve contributed. No upload without download. Build in export capabilities from day one. If you close a system, support data rescue. Provide one financial quarter’s notice between announcing the shutdown and destroying any user-contributed content, public or private, and offer data export during this period. And beyond that three months? Make user-contributed content available for media-cost purchase for one year after shutdown.
#cloud #data #services #to:fb
archive.org #22373

SUMO and migrating large amounts of data

http://embrangler.com/2010/12/migrating-large-amounts-of-data/
Data migration is a complicated problem. It’s one of those things nobody wants to do, because it’s mostly tedious, not engaging, and also important to get right. Often times you have to compromise. I’ll talk a bit about the history of SUMO’s data, decisions the product and development team have made regarding its migration, and things I’ve learned from the experience.
#data #imported:pinboard #migration #mozilla #sumo #webdev
archive.org #33015

SUMO and migrating large amounts of data

http://embrangler.com/2010/12/migrating-large-amounts-of-data
Data migration is a complicated problem. It’s one of those things nobody wants to do, because it’s mostly tedious, not engaging, and also important to get right. Often times you have to compromise. I’ll talk a bit about the history of SUMO’s data, decisions the product and development team have made regarding its migration, and things I’ve learned from the experience.
#data #migration #mozilla #sumo #webdev
archive.org #21373

CouchDB "Joins" - Christopher Lenz

http://www.cmlenz.net/archives/2007/10/couchdb-joins
Obvious to Damien, but not at all obvious to the rest of us: it's fairly simple to make a view that includes both the content of the blog post document, and the content of all the comments associated with that post. The way you do that is by using complex keys. Until now we've been using simple string values for the view keys, but in fact they can be arbitrary JSON values, so let's make some use of that:
#couchdb #data #database #nosql #webdev
archive.org #19656

Steven Poole: Whatever made you think it was your data anyway?

http://stevenpoole.net/blog/whatever-made-you-think-it-was-your-data-anyway
In case it helps, I hereby declare the following iron law of “free” internet services: If you’re not paying for something, you have no reason to expect it to be there tomorrow. This is an important corollary to the law “If you’re not paying for something, you’re not a customer; you’re the product being sold”.
#cloud #data #fuckthecloud #googlereader #ifttt #social #web20
archive.org #19513

Don't use MongoDB | Hacker News

http://news.ycombinator.com/item?id=3202081
From CTO of 10gen First, I tried to find any client of ours with a track record like this and have been unsuccessful. I personally have looked at every single customer case that’s every come in (there are about 1600 of them) and cannot match this story to any of them. I am confused as to the origin here, so answers cannot be complete in some cases.
#data #database #mongodb #nosql #webdev
archive.org #19494

Detecting Support for data: URIs | Weston Ruter

http://weston.ruter.net/2009/05/07/detecting-support-for-data-uris
The data: URI scheme is now supported by the most current version of every major browser, including Internet Explorer. Because of this I wanted to use CSS background images encoded with data: URIs in a current project at Shepherd Interactive. Why? The first rule of High Performance Websites is to Minimize HTTP Requests. By storing background images directly in the stylesheet only one HTTP request is then necessary to fetch the stylesheet and the images all at once. Furthermore, by giving that stylesheet a far-future cache expiration date the browser will never need to request it again.
#css #data #webdev
archive.org #15444

Persistent Trees in git, Clojure and CouchDB « EclipseSource Blog

http://eclipsesource.com/blogs/2009/12/13/persistent-trees-in-git-clojure-and-couchdb-data-structure-convergence
"This is a tale of three images. I found these images while investigating the internals of several different applications. There are some really neat software projects emerging at the moment, and as a developer I always find it interesting to take a look at the implementation details, because there is often a lot to be learned. It’s not always something you might need right now, but maybe a few years down the line you may be confronted with a similar problem. Plus – in my opinion – knowing a bit about the internals of a program helps reasoning about its behaviour."
#clojure #couchdb #data #distributed #git #persistence #trees
archive.org #13641

NoSQL vs. RDBMS: Let the flames begin! – stu.mp

http://stu.mp/2010/03/nosql-vs-rdbms-let-the-flames-begin.html
"I guess what I’m saying is that my decision to use NoSQL, and I’m guessing others’ decisions to do so, has less to do with the fact that we can’t squeeze a few thousand writes a second out of MySQL and more to do with management and cost overhead. NoSQL solutions allow us to serve absurd amounts of data for a really, really low price. I’m happy to put my $/write, $/read, and $/GB numbers for my NoSQL setup against anyone’s RDBMS numbers. We’re not nearly as dumb as everyone thinks we are; I promise."
#costs #data #databases #expenses #nosql #rdbms
archive.org #13103

Edward Vielmetti - Downtown parking information by telephone project: my part is over

http://vielmetti.typepad.com/vacuum/2009/05/downtown-parking-project-over.html
"Sorry I ever brought it up, sorry for any hurt feelings. It was wrong of me to think that I could make any progress without the support of the organization providing the data, and wrong of me to press on regardless of the cost. Someone else will have to care - I give up. It's just not worth it and it's bad for my health to care and to spend too much time around parking structure fumes."
#annarbor #civics #data #foia #opengovernment #openness
archive.org #12234

Edward Vielmetti - I am not an "information activist" or a "civic minded data freak", thank you very much

http://vielmetti.typepad.com/vacuum/2009/05/i-am-not-an-information-activist-or-a-civic-minded-data-freak-thank-you-very-much.html
"So if I ask for that page of numbers as the spreadsheet it was originally, and a public records officer for a public body gives it to me as a bitmap, I'm going to ask again and keep asking publicly and perhaps uncomfortably until we both understand that this is a routine request and that compliance with routine requests is much easier than making everything an exception - because maybe, just maybe, I can replace that FOIA letter with a tiny shell script and we can just go about our business."
#annarbor #civics #data #foia #government #opengovernment #openness
archive.org #12233

Monkeying with the Lizard Feeder

http://surfmind.com/muzings?cpage=1&p=237#comment-312760
"The call to action is to build on the Lizard Feeder concept with anything from ideas to pixels to code. I hacked up a grease monkey script that weights the data-source checkbox & label on the left according to the volume of flow. This provides a bit of focus + context for the recent stream. The implementation is a minimal, bit-of-an-evening starter kit (MPL) for more interesting creations."
#activitystreams #data #feeds #lizardfeeder #mozilla #visualization
archive.org #11947

getluky.net » A Warning About the Real Cost of Microformats

http://getluky.net/2009/01/08/a-warning-about-the-real-cost-of-microformats
"Anyway, here’s the question I want to put into the reader’s mind: should one spend time and effort making a frontend into an informal API through microformats, or to instead spend it on building a fully supported API or data publishing system that exists and operates separately? I think my stance is clear - i’m not against the theory of microformats, but i’m certainly going to differ with anyone who thinks it’s practical."
#apis #data #microformats #web20 #webdev #xhtml
archive.org #11854

the ryan king » Introducing Conveyor

http://theryanking.com/entries/2008/02/26/introducing-conveyor
"The future for large websites’ data storage is likely a collection of special purpose data stores: GFS/MapReduce for batch jobs, inverted indexes for search and fast retrieval of small result sets and relational databases for smaller datasets which nee
#data #databases #webdev
archive.org #10453