Posted by Matt on June 12, 2011 – 12:37 am
Heres another little function I’m using for the new Musotik. Don’t mind the sloppy code. It searches Google for mp3 indexes and returns the results in an array. Try it out here
function getindexes($q) {
$api = "SET YOUR GOOGLE API KEY HERE";
$cx = "SET YOUR GOOGLE CX VAR HERE";
$q = '-inurl:htm -inurl:html -inurl:php -inurl:asp -inurl:doc -inurl:pdf -inurl:shtml -inurl:txt AND (“index of|"last modified"|"parent of") AND mp3 -ringtone -lyric -playlist AND '.$q;
$q=urlencode($q);
$url="https://www.googleapis.com/customsearch/v1?key=$api&cx=$cx&q=$q&alt=atom";
$xml = new XMLReader();
$xml->open($url);
$assoc = xml2assoc($xml);
$xml->close();
$entries = array();
foreach($assoc[feed][0] as $node) {
foreach($node[entry] as $entry) {
$entries[link][] = $entry[value][id][0][value];
$entries[title][] = $entry[value][title][0][value];
}
}
return $entries;
}My return array is a little wonky here, but you can return it how ever you want. Below is a utility function used above taken from php.net
function xml2assoc($xml) {
$assoc = null;
while($xml->read()){
switch ($xml->nodeType) {
case XMLReader::END_ELEMENT: return $assoc;
case XMLReader::ELEMENT:
$assoc[$xml->name][] = array('value' => $xml->isEmptyElement ? '' : xml2assoc($xml));
if($xml->hasAttributes){
$el =& $assoc[$xml->name][count($assoc[$xml->name]) - 1];
while($xml->moveToNextAttribute()) $el['attributes'][$xml->name] = $xml->value;
}
break;
case XMLReader::TEXT:
case XMLReader::CDATA: $assoc .= $xml->value;
}
}
return $assoc;
}
Posted by Matt on March 28, 2011 – 12:06 am
Filed under code, geek, projects
Tagged as curl, php, skreemr
Heres a bit of a larger project I’m working on that will give you the first 10 skreemr mp3 links for a search.
<?php
$q = urlencode($_GET['q']);
$skreemr="http://skreemr.com/results.jsp?q=$q";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $skreemr);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, TRUE);
$output = curl_exec($ch);
curl_close($ch);
$re1='.*?'; # Non-greedy match on filler
$re2='((?:http|https)(?::\\/{2}[\\w]+)(?:[\\/|\\.]?)(?:[^\\s"]*))'; # HTTP URL 1
if ($c=preg_match_all ("/".$re1.$re2."/is", $output, $matches)) {
foreach ($matches[1] as $url) {
if (preg_match('/mp3/', $url)) {
echo "<a href=\" $url\" target=new>$url</a><br />";
}
}
}
?>
Posted by Matt on December 13, 2010 – 3:52 pm
So I’ve decided that I’m going to write/find a web crawler to find mp3s/ogg etc.. Then rewrite Musotik from scratch. It’ll also crawl torrent sites. I’m working with Sphider right now. It may become the base of my crawler.
After I get the spider wrote, I’ll run it on 3 of the cluster boxes, updating 1 mysql db. The other box will be the main webserver and database server.
- 12/13/10 2am – Started ripping apart the Sphider script.
- 12/13/10 2:30am – 3 nodes are indexing to the same DB. Testing with Digg, PirateBay, and Drawgasmic
- 12/13/10 3:45am – Going to let it index.. here what I have so far:
- —-Currently in database: 16 sites, 4964 links, 0 categories and 103853 keywords.
- 12/13/10 4pm – let it run all night/day.
- —-Currently in database: 16 sites, 32603 links, 0 categories and 278085 keywords.
So its pretty slow with Sphider. I also don’t need everything that Sphider does. I’m trying to decide whether I should write something from scratch or modify Sphider.
My next post will be about that, and probably heavy with PHP code.
Posted by Matt on December 10, 2010 – 10:22 am
So I came into possession of 4 p4 1.5-1.7Ghz Dell boxes. Each machine has 256Megs of ram, a cd-writer and a floppy drive. Here is a break down of how I try to turn them into a cluster, and what I decide to do with this said monster. 
- 12/9/10 6pm – The computers arrive at our apartment and I’m excited they’re all the same size and almost the same specs. We go to dinner. I think about them during the entire meal.
- 12/9/10 10pm – I come home and boot one up. It boots into XP just fine, the cdrom may be jacked up on this one. I try to install extra memory I have laying around. No go. We’re stuck at 256Megs. I start reading about clustering with Ubuntu on the webz.
- 12/9/10 11pm – Started downloading Ubuntu server 10.10 and configuring the router DNS for the 4 nodes. I’ll be using the IP block of 192.168.2.100-105. Still very unsure of how this all works. Back to reading while this cd downloads and burns.
- 12/10/10 12am – Started install Ubuntu Server 10.10 on the first node. One of the 1.7Ghz.
- 12/10/10 12:30am – Burnt a DVD… The cluster nodes only have CDs. Burning a CDR.
- 12/10/10 12:45am – Trying to burn a CDR with one of the node machines, but it appears to be locked up at 12%. Canceling and trying again. Copying the ISO to the local machine to burn instead of over the network.
- 12/10/10 2am – Burn and install successful, but I installed Ubuntu with the eucalyptus and landscape only to find out they’re intense on the poor lil P4 CPU. Reinstalling… meh.
- 12/10/10 4:45am – Installed Ubuntu Server on 2 machines. The cd-rom in the one 1.7Ghz machine is definitely bad. I’ll have to swap out one from the another machine to install. No biggie. So I haven’t even started on the clustering part. But I’ll try something out once I get both the machines configured the same. And updated.
- 12/10/10 6:30am – Got Folding@home running on the first 2 nodes, installing Ubuntu on the 3rd. I’m using https://help.ubuntu.com/community/MpichCluster as a guide.
- 12/10/10 10:00pm – Went remote and made the mistake of trying to edit the network. Took all 4 machines down somehow. Now I have to wait till tonight to fix it locally. meh.
So I managed to get fold@home running on three nodes in about 12 hours. More to come in the next post.
Posted by Matt on January 27, 2010 – 6:41 pm
Drawgasmic Art Exhibition – The Art, Illustration, and Design Compendium. Happening in July 2010 at the Mad Art Gallery.
The Drawgasmic exhibition and book is an effort to expose, promote, and push forward the work of international independent artists, illustrators, and designers working today. We, as curators, are trying to create the sort of experience we feel is missing in the in the all to often pompous and institutionalized art world. An experience that combines the talent of hundreds of great artists with a mash-up of unique performances.
We are excited to be able to document the entire collection in a quality, limited edition coffee table book. The book will exisit as a historical marker of the unique styles and concepts being created by living and working artists of 2010. This book is being released in conjunction with the exhibition in July 2010.
Posted by Matt on December 12, 2009 – 7:49 pm
I’ve brought musotik back to life. I’ll be working on/rewriting her as I get time. She lives here now.
