May 132013
 

I haven’t been updating the blog lately because I have been fervently involved in building a Bitcoin machine that can buy, sell, and integrate third party bitcoin businesses right from a standing kiosk. My brother and I will be showcasing the machine at the Bitcoin 2013 conference for everyone to use. More information about the product can be found at our website here. I hope to begin updating frequently once again! Stay tuned!

Robocoin

Jan 252013
 

Visualize and analyze the Bitcoin block chain with the most advanced publicly available web-based analysis tool to date. By provably combining related bitcoin addresses into owners and constructing a visualized payment network of these owners, the Block Viewer provides new and more useful information for those looking for a better way to track payments and people.

Block Viewer

The side bar provides additional information about the owner, including links to time-division snapshots of the owner network. Click on a date and follow transaction chains through the day. This feature is still experimental but can be an extremely powerful way to quickly cut through payment obfuscation techniques.

The Block Viewer is about 1.5 weeks behind at the moment (the time division graphs are almost a month behind), but the server has been re-enabled and is updating the graphs right now. Submit addresses you know to have an identity with the “Identify” link in the menu bar and make the tool more useful by labeling owners! (Right now it scrapes Bitcoin-OTC and BitcoinTalk forum signatures to initially seed the labels).

Catch me on the forums for questions and concerns. Check out the source on Github. Enjoy!

Dec 182012
 

After the drama surrounding Instagram’s new Terms of Usage, all of us at Cloudsnap thought it would be pretty cool to give people a choice in where they store their photos. As a result, we created a tool that migrated their accounts from Instagram to Flickr! After a good day of crunching, we posted our application up on Hacker News where it held the #1 spot for several hours. Building the application was pretty straightforward, though some idiosyncrasies with the Flickr API made it a bit more frustrating then it had to be – particularly their image upload endpoint. The API was XML, OAuth 1, and the image upload required a multi-part post – a deadly combination. Their API claims to have a full REST implementation, though GET and POST requests can be used interchangeably on almost all of their endpoints which was weird.

After about 14 hours of our service going online, we have already helped over 1,000 users move over 300,000 photos to Flickr. We used Iron.Io to help us scale each user migration process, which so far has handled the move beautifully. The only issue with them as it stands is that we had to compile our project at home using the same infrastructure they use as they merely execute and not attempt to compile the binaries them on their end.

Anyway, check it out, and Free The Photos! -> http://freethephotos.com/

Screen Shot 2012-12-19 at 8.51.07 AM

 Posted by at 12:00 am
Oct 302012
 

Hubot is a program that sits in on chat rooms and can prove to be incredibly useful or incredibly useless depending on the implementation – from alerting you of recent changes to your company github repository to dropping “that’s what she said’ jokes at unsuspecting double entendres. Created by Github, they claim that “Hubot is your company’s robot. Install him in your company to dramatically improve and reduce employee efficiency.” In this post, I just want to quickly demonstrate a plugin I wrote one night that allows people in the room play chess:

# Description:
#   Play a game of chess!
#
# Dependencies:
#   "chess": "0.1.3"
#
# Configuration:
#   None
#
# Commands:
#   hubot chess me - Creates a new game between yourself and another person in the room
#   hubot chess status - Gets the current state of the board
#   hubot chess move <to> - Moves a piece to the coordinate position using standard chess notation
#
# Author:
#   thallium205
#
 
Chess = require 'chess'
 
module.exports = (robot) ->
  robot.respond /chess me$/i, (msg) ->
    robot.brain.data.chess = Chess.create()
    boardToFen robot.brain.data.chess.getStatus(), (status, fen) ->
      msg.send 'White\'s turn.'
      msg.send 'http://chessup.net/php/pictureParser.php?fen=' + fen + '&coord=on&.png'
  robot.respond /chess status/i, (msg) ->
    try
      boardToFen robot.brain.data.chess.getStatus(), (status, fen) ->
        msg.send status
        msg.send 'http://chessup.net/php/pictureParser.php?fen=' + fen + '&coord=on&.png'
    catch e
      msg.send e
 
  robot.respond /chess move (.*)/i, (msg) ->
    try
      robot.brain.data.chess.move msg.match[1]
      boardToFen robot.brain.data.chess.getStatus(), (status, fen) ->
        msg.send status
        msg.send 'http://chessup.net/php/pictureParser.php?fen=' + fen + '&coord=on&.png'
    catch e
      msg.send e
 
boardToFen = (status, callback) ->
  fen = [[],[],[],[],[],[],[],[]]
  blank = 0
  lastRank = 0
  for square in status.board.squares
    if lastRank isnt square.rank
      if blank isnt 0
        fen[lastRank - 1].push(blank)
        blank = 0        
    if square.piece is null
      blank = blank + 1
    else
      if square.piece.type is 'pawn'
        if blank is 0          
          fen[square.rank - 1].push(if square.piece.side.name is 'white' then 'P' else 'p')
        else
          fen[square.rank - 1].push(blank)
          fen[square.rank - 1].push(if square.piece.side.name is 'white' then 'P' else 'p')
          blank = 0
      else
        if blank is 0
          fen[square.rank - 1].push(if square.piece.side.name is 'white' then square.piece.notation.toUpperCase() else square.piece.notation.toLowerCase())
        else
          fen[square.rank - 1].push(blank)
          fen[square.rank - 1].push(if square.piece.side.name is 'white' then square.piece.notation.toUpperCase() else square.piece.notation.toLowerCase())
          blank = 0
    lastRank = square.rank
  for rank in fen
    rank = rank.join()
  fen = fen.reverse().join('/').replace(/,/g,'')
 
  msg = ''
  if status.isCheck
    msg += 'Check! '
  if status.isCheckmate
    msg += 'Checkmate! '
  if status.isRepetition
    msg += 'Threefold Repetition!  A draw can be called. '
  if status.isStalemate
    msg += 'Stalemate! '
  if Object.keys(status.notatedMoves).length > 0
    if status.notatedMoves[Object.keys(status.notatedMoves)[0]].src.piece.side.name is 'white' then msg += 'White\'s turn. ' else msg += 'Black\'s turn.'
 
  callback msg, fen

The hard work was already done for me by a great javascript chess library ‘chess’, though I am no stranger to creating a chess board from scratch so I can definitely appreciate the thankless work that went into that. The only part that had to be done on my part was to iterate through the chess board and convert the piece positions to FEN notation which is a format that gives a lot of portability to representing a chess board. From there, the string was merely plugged into a website that generates board states from it, and then the plugin returns the picture’s url to the chatroom:

I feel creating a plugin is pretty straight forward – you can check out their huge repository of already approved scripts here. Submitting a new script requires a pull request to their repository, which is what this plugin is waiting for -> https://github.com/github/hubot-scripts/pull/620

Oct 232012
 

Importing a Neo4j database using Gephi’s user interface is as trivial as downloading the plugin and selecting the graph.db folder. However, programmatically interacting with a 3rd party plugin requires a couple steps that are not immediately apparent.

1) Download the Gephi Toolkit and include that in your build. Download the Neo4j Plugin as well.

2) In order to use the plugin, one must first download UnpackNBM and execute the command ‘java -jar UnpackNBM.jar org-gephi-neo4j-plugin.nbm’ which extracts the needed jar files that are then used and included in the project.

3) Write the importer:

      LOG.info("Begin Building GEXF from Neo4j");
      LOG.info("Fetching graph...");
      // Get the database and the owner index 
      graphDb = new EmbeddedGraphDatabase (dbPath);
      owned_addresses = graphDb.index().forNodes(OWNED_ADDRESS_HASH);
 
      // Find an node id.  This will be the start location of the graph traversal and is guarenteed to be part of the graph.
      final long ownerId = getKnownOwnerId();
 
      // Import by traversing the entire ownership network along the "transfers" edges
      LOG.info("Importing Ownership Network from Neo4j Database...");
      final Collection<RelationshipDescription> relationshipDescription = new ArrayList<RelationshipDescription>();
      relationshipDescription.add(new RelationshipDescription(GraphBuilder.OwnerRelTypes.transfers, Direction.BOTH));
      final Neo4jImporter importer = new Neo4jImporterImpl();      
 
      // Load the graph in memory
      importer.importDatabase(graphDb, ownerId, TraversalOrder.BREADTH_FIRST, Integer.MAX_VALUE, relationshipDescription); 
 
      // Grab the graph that was loaded from the importer      
      final ProjectController projectController = Lookup.getDefault().lookup(ProjectController.class);
      final Workspace workspace = projectController.getCurrentWorkspace();
      final GraphModel graph = Lookup.getDefault().lookup(GraphController.class).getModel(workspace);   
      LOG.info("Graph Imported.  Nodes: " + graph.getDirectedGraph().getNodeCount() + "Edges: " + graph.getDirectedGraph().getEdgeCount());
 
      // Layout
      final ForceAtlas2 layout = new ForceAtlas2(new ForceAtlas2Builder());
      layout.setGraphModel(graph);
                .
                .
                .

I opened up a question to verify the validity of the process on their forum without any feedback, but the good news is that the bug request I submitted has been resolved which allows me to download the nightly build of the toolkit to finish up the last step of the visualizer. I really wish I had more time to work on all this to report back with more progress. Happy importing!

 Posted by at 12:00 am
Oct 092012
 

As I adjust to a new life schedule the development time put into the project at the moment is very little (though this will be changing as I continue to adjust!). However, the biggest problem that has stopped progress happens during graph export: whenever I am finished manipulating a subsection of the ownership network in Gephi, either in the toolkit or the GUI, I am not able to export it to any format due to this bug I have submitted to the community: https://github.com/gephi/gephi/issues/679

Hopefully I can get a moment to find the (trivial) problem this weekend, but that is unlikely as I have to prepare for a paper presentation in class next week talking about a new scaling method for the block chain using an authenticated data structure (some information can be found here -> https://bitcointalk.org/index.php?topic=88208.0;all

In the meantime, I present to you two small samplers of what is to come. First, check out this screen shot of the ownership network organized by the OpenORD graph algorithm and conforming to the formatting scheme laid out in my previous post for all of September, 2012 (click on the image to download a pdf to see the vector quality visualization – that is, if your computer is up to the challenge):

Now, imagine this graph in a zoomable, searchable, and information rich user interface. This is made possible by modifying the fantastic gexf-js library:

Each time a user enters a bitcoin address in the search bar, it will:

- Zoom to the ownership node in the latest ownership snapshot it existed in (the above screenshot was for a month – I suspect the graph will need to be limited to weekly or even daily snapshots to keep it less crowded and easier to organize)
- The amount of Bitcoin that owner has sent and received and at what times.
- The name of the owner if it has been identified to a real world identity such as a username.
- All the other addresses provably associated with it.

Here is the query that is executed from nodejs:

var 
	ownerIdQuery = [
		'START addr = node:owned_addr_hashes(owned_addr_hash={addr})',
		'MATCH addr <- [:owns] - owner',
		'RETURN ID(owner) AS id'].join('\n'),
	sentQuery = [
		'START addr = node:owned_addr_hashes(owned_addr_hash={addr})',
		'MATCH addr <- [:owns] - owner - [to:transfers] -> receiver',
		'RETURN to.time? AS time, to.value AS amount'].join('\n'),
	receivedQuery = [
		'START addr = node:owned_addr_hashes(owned_addr_hash={addr})',
		'MATCH addr <- [:owns] - owner <- [from:transfers] - sender',
		'RETURN  from.time? AS time, from.value AS amount'].join('\n'),
	identifierQuery = [
		'START addr = node:owned_addr_hashes(owned_addr_hash={addr})',
		'MATCH addr <- [:owns] - owner <- [id?:identifies] - identifyingAddr',
		'RETURN id.name AS name, id.time AS time, id.source AS source, id.contributor AS contributor, identifyingAddr.addr AS address'].join('\n'),
	addressesQuery = [
		'START addr = node:owned_addr_hashes(owned_addr_hash={addr})',
		'MATCH addr <- [:owns] - owner - [:owns] -> owned',
		'RETURN owned.addr AS address'].join('\n'),
	params = {
		addr: req.params.id,
		},
	results = {};

These cypher queries are from Neo4j’s 1.7.2 engine, and their newly released engine, 1.8, allows for the chaining of queries when I asked about it here -> http://stackoverflow.com/questions/12269009/returning-two-aggregates-in-a-single-cypher-query I hope to implement the new version and clean up these queries more with their more powerful query language.

I swear I’m doing stuff – life just gets in the way sometimes and slows these things down. Soon!

Sep 252012
 

It’s been a while since I updated with the latest information regarding blockviewer.com. Currently, the site is in read-only mode as I slowly transition to a new visualization that I hope will be both useful and revealing regarding the Bitcoin network. What is currently in the works right now is a beautiful graph representing the entire ownership network, where each element in it represents data, including:

Node Label – If an owner has been identified to a real world entity such as a bitcointalk.org username, nodes will be tagged with this information. Otherwise, they will be tagged with a unique identifier.
Node Size – The amount of inbound/outbound transactions that have been sent to and from a particular owner will be represented with how large a node is.
Node Color – Hotter colors will represent recent activity sent from the owner while a cooler color will represent older activity.
Edge Size – The amount of BTC being sent, where larger lines represent large amounts while smaller lines represent smaller amounts.
Edge Color – Hotter colors will again represent recent transactions in time while cooler colors will represent older activity.

The entire graph will be explorable similar to how Google Maps works – complete with zooming and searching. However, as I have begun to do this, I have reached an impasse – the ownership network, as it stands, has over 3 million nodes and 3 million unorganized edges. Trying to organize such a large graph into something visually useful (instead of some hugely unorganized blob) is unheard of using a conventional computer. I have two options at the moment: 1) Filter the ownership network to only include nodes that have resembled recent activity or 2) Find a better computer or a more efficient method.

I am in the process exploring both options at the moment. By using the UNR Research Grid, it is possible to gain access to a better computer. I only need to convince a professor that this is a worthwhile project so that I can gain access to it and to also programatically set up the Gephi Toolkit in such a way that it will scale to such a large amount of machines. I am also looking into a Force Atlas 2 library that uses Nvidia’s CUDA to run the algorithm instead of CPU cores. I have no experience with CUDA programming, but it appears to be a feasible option.

If these two methods don’t work, I will filter the network by monthly snapshots – where a user could select a month and explore the activity of the network at that moment in time. This may prove to be more useful (and more practical) in the long run anyway. I hate to say this, but once again, stay tuned!

Sep 112012
 

After a lot of good feedback in the forums I have decided to take a radically different approach to visualizing the bitcoin block chain. The fundamental problem with the visualizer as it stands right now is that it does not take into account closely related owners and the probability that they are in fact also the same person. It also prohibits the end user from really exploring the macro relationships between other owners. As a result, the new visualization I am working on will be more like how google maps works where a fully zoomed out maps allows you see continents, major countries, etc. As you zoom in, the map gets more detailed. This concept will be applied to the visualization – where a fully zoomed out map will let you see the largest owners – ie: mt gox, satoshi dice, etc. As you zoom in, you see the smaller owners. You will only be able to search by address, which would be like typing in a zip code into google maps where it will zoom in right to the owner but you can see all the nodes around it.

Circle radius will be denoted by the amount of times they have sent and received from other owners (they will also be labelled if they were identified to real-life entities like screen names). Lines will visualize two 2 dimensions: thickness will be amount sent and color will be how recent it was sent where hotter colors are very recent while cooler colors are old.

I am currently rebuilding the owner network to include the times when people sent money to other owners, so that will take a couple days to build. I have also been adding more websites the program scrapes to associate owners to real life screen names. I need to figure out a good library I can use which can help me display several of these “layers” (the top being the least detailed, the bottom being the most detailed) in an efficient manner and stitch them together.

Below is a time lapse organization of a small subsection of the ownership network – with the wikileaks owner being at the start node. Each one of these tiny nodes can provably represent 1 to over 500,000 bitcoin addresses – each edge represents a payment to another owner. A group of them can represent potentially millions of addresses. Watch how closely grouped owner nodes begin to materialize – a truly incredible grouping that totally blasts the anonymity of the network out of the water. Can you spot Mt. Gox – bitcoin’s largest exchange while listening to some Maple Leaf Rag? (I apologize for the video quality in advance)

The final product can be found here!

STAY TUNED!

 Posted by at 12:00 am
Aug 282012
 

This is a cross post from my announcement on the bitcoin forums about my latest project! It should be noted that this is definitely only the beginning and I plan on actively developing this application – the queries that can be created from the ownership network have the potential to be extremely powerful… it’s just my job to show people why! The visualizer is only a very rough release intended for bitcoin power users – cleaning the client interface (and adding awesome amounts of derived data that isn’t being shown) is a priority. The launch has been going relatively smooth – but after a pervasive bug that resulted in duplicate owners and fragmented same-owner sub-graphs spotted by one of the users, I can safely say the data is now correct.

http://blockviewer.com

I would like to announce the start of a new project that attempts to not only give everyone an interactive way to visualize and interact with the block chain, but to also have an easy and fun way to determine addresses owned by the same person and how that person interacts with others: http://blockviewer.com/#/owns/1HB5XMLmzFVj8ALj6mfBsbifRoD4miY36v

In the above screenshot, the owner in the middle has 43 addresses that are all owned by him:

In addition to knowing all linked addresses, a high level network of “owners” has been constructed which can also track payments between other high level owner addresses as well:

Here we can see this owner entity received 183 payments from others, and sent 27 payments to other owners. Expanding one of the owners out of the folders, it is revealed that he sent 127.37 BTC to another person (who happens to control 5 addresses).

This application exploits the weakness identified in Satoshi’s original paper (pg.6 sec. 10) where he stated that “Some linking is still unavoidable with multi-input transactions, which necessarily reveal that their inputs were owned by the same owner. The risk is that if the owner of a key is revealed, linking could reveal other transactions that belonged to the same owner.” This application has constructed this high level owner network on top of the normal block chain, which is also traversable by simply clicking:

If you would like more information on how I did this, check out my use case here: http://toolongdidntread.com/bitcoin/the-bitcoin-social-network-part-1/ (Note: this database does not explicitly identify forum users yet. I will be enabling that information later on.)

The block chain is a graph, why are we not displaying it as such? If you would like to your public address against it, simply type in your address in the search box and select “Owner Address”. If all worked well, it will have all the other addresses that have been linked to you through the weakness, and you can see how well it determined where you sent BTC to other people.

KNOWN ISSUES:
There are so many it’s tough to even begin. This is definitely a “Use At Your Own Risk” as it is bound to crash, get turned off, shut down at any time for maintenance, etc.
The queries stop after 5000 nodes have been returned to protect the server. I hope to implement pagination soon.
The API allows you to download subgraphs in a standardized .gexf data format that can be used by graph visualization tools like Gephi. This will still stop queries after returning 5000 nodes.
The visualizer and client-side javascript is a mess. I will be cleaning it up quite a bit.
It’s not very user friendly at the moment, nodes are difficult to click on, and the documentation is pretty ugly. I wanted to get this out anyway so power users could check it out.
At the moment, I am using blockchain.info’s API that rate limits me which results in data that is probably like a day old. If piuk has it in his heart to forgive me for crashing his server, I would love another API key

Anyway, I built this because its so much easier to track payments using the block chain as a graph instead of pulling up HUGE lists that are on blockchain.info. The database is colossal, and its only getting bigger. Think of this as more of a sleuthing tool I can’t wait to hear your comments, suggestions, ideas, and the inevitable troll who says its a scam. Thanks!

Aug 142012
 

… but I found a problem with the high level graph builder that prevents new nodes from appending address owners. So the deadline is next Sunday – extremely disappointing as I really want to submit this to GraphConnect 2012 ASAP -> http://www.graphconnect.com/ … In the meantime, check out some screen shots! Each node will be fully clickable and is bursting with information as you hover your mouse over them. You will be able to drag them, expand them, and ungroup them from the folders that keep the visualization clean. Its API produces standard GEXF data files that allow programs like Gephi and http://sigmajs.org/ to interpret it, and the first feature after release will allow you to share your graph creation with others by a link! It will take a week alone to rebuild the graph with the changes, so stay tuned!

The high level owner graph is at the bottom and the low level block chain graph is at the top:

Here’s another example: