Watching every episode of The X-Files in high school on Netflix DVDs that came in the mail (remember those?) seemed like the thing to do. It was a great show, with 9 seasons of 20+ episodes a piece. So, it only seemed fair to provide a robot friend with the same experience.

I’m currently running NeuralTalk2, which is truly wonderful open source image captioning code consisting of convolutional and recurrent neural networks. The software requires a GPU to train models, so I’m running it on an Amazon Web Services GPU server instance. At ~50 cents per hour, it’s a lot more expensive than Netflix.

Andrej Karpathy wrote NeuralTalk2 in Torch, which is based in Lua, and it requires a lot of dependencies. However, it was a lot easier to set up than the Deep Dream code I experimented with over the summer.

The training process has involved a lot of trial and error. The learning process seems to just halt sometimes, and the machine often wants to issue the same caption for every image.

Rather than training the machine with an image caption set, I trained it with dialogue from subtitles and matching frames extracted at 10 second intervals from every episode of The X-Files. This is just an experiment, and I’m not expecting stellar results.

That said, the robot is already spitting out some pretty weird and genuinely creepy lines. I can’t wait until I have a version that’s trained well enough to feed in new images and get varied results.

Below is my code, which includes encode and decode functions.

def encode(u):
	# init tracking vars
	enc = list()
	cur = list(u[0])
	# iterate through string
	for i in range(1, len(u)):
		if u[i] == u[i-1]:
			cur.append(u[i])
		else:
			enc.append((len(cur), u[i-1]))
			cur = list(u[i])
	# handle last character
	enc.append((len(cur), cur[0]))

	def rep(tup):
		# encode using backtick as delimeter
		count, char = tup
		if count == 1:
			return char
		else:
			return "%s`%i`" % (char, count)

	return ''.join(map(rep, enc))

def decode(e):
	# init tracking vars
	result = list()
	cur_num = list()
	# switch var
	on_num = False
	# iterate through encoded string
	for i, char in enumerate(e):
		# if delimeter found...
		if char == "`":
			# switch on/off
			on_num = not on_num
			# if closing delimeter
			if not on_num:
				result.append(
					rep_char*int(''.join(cur_num))
				)
				cur_num = list()
			# if opening delimeter
			elif on_num and i > 0:
				# repeated char is last
				# added to result
				rep_char = result.pop()
		# if not delimeter and not on number
		elif not on_num:
			result.append(char)
		# if not delimeter and on number
		else:
			cur_num.append(char)
	return ''.join(result)



if __name__ == '__main__':
	import sys
	to_encode = sys.argv[1]
	encoded = encode(to_encode)
	decoded = decode(encoded)
	print encoded
	print decoded
	assert to_encode == decoded