Parse quicktime movies with ruby

We had a problem with our flash movie player: most videos played fine,
but for some files the whole video had to be transferred before it started
to play. What was going on here?

It turned out that the quicktime format’s header, the “moov” atom, can
actually be at the end of the file – presumably because it is
convenient for video editing software. There’s also a fix for this
problem: newer ffmpeg distributions come with a small helper called
qt-faststart that can fix this kind of problem.

Anyway, “qt-faststart” needs to copy the whole movie to do this, so
how do we find out if it is really necessary? Furthermore, we’ve
already got a decent number of video files on our platform we’d need to
check for streamability.

One solution is to use a command line tool to check for this. Before I
heard of qt-faststart and before I knew how to parse quicktime movies,
I tried to (ab)use mplayer for this:

mplayer -msglevel all=7 -frames 1 -vo null -ao null $FILE | \
sed -n 's/^MOV: Movie header: start: \(.*\) end:.*/\1/p'

This starts playing a the movie file with mplayer, but just one frame,
discarding the video and audio generated, with lots of debug output
going to stdout. Another solution is to use qt-faststart, sending its
output to /dev/null and parsing the messages it spits out.

On the other hand, “qt-faststart” is not such a complicated program:
it’s just 316 lines of C code. So I thought this functionality should
be easy to port to ruby. And it turned out that the quicktime format
is not that difficult to parse, at least for the very basic
information we need to get.

A quicktime movie is made up of blocks called “atoms”. Every atom
begins with an 8 byte header, the first 4 bytes determine the size of
the atom including the header, a 32 bit unsigned integer, while the
second 4 bytes make up a 4 letter ASCII string for the atom type. As
we only want to know where the “moov” atom is we don’t need to know
what the contents of an atom look like.

So without further ado, here’s the method we use now to determine if a
video file is streamable:

def streamable?(file)
  atoms = %w(free junk mdat moov pnot skip wide PICT ftyp cmov stco co64)
  File.open(file) do |f|
    while !f.eof?
      # the size is an unsigned 32bit integer, big-endian AKA network byte order
      # the type is 4 ascii characters
      size, type = f.read(8).unpack("Na4")
      raise ArgumentError, "unknown atom type #{type} in #{file} at #{f.pos}" unless atoms.include?(type)
      return f.pos < 0xff if type == "moov"
      f.seek(size - 8, IO::SEEK_CUR)
    end
    raise ArgumentError, "no moov atom found"
  end
rescue
  false
end

looking old with new

Just spent the last few minutes figuring out what broke my tests, until I found what you can call a really bad idea:

named_scope :new, :conditions => { :state => 'new' }

Never thought that one could get TOO used to “expressing things naturally”… but of course, it’s just another example of the keyword trap.

Rendering from inside your models

There are few, but special occasions when you want to render templates directly inside a ActiveRecord model, circumventing the usual model-controller-view process. For example, when a contract is closed on the MPX platform, the resulting contract PDFs have to be rendered and safely stored without user interaction. The following class demonstrates how this can be accomplished.

class Contract < ActiveRecord::Base

  before_validation_on_create :create_pdf

  class ViewRenderer < ActionView::Base
    include *helper_modules
    include ApplicationHelper, SomeOtherHelper

    include ActionController::UrlWriter
    default_url_options[:host] = 'www.example.com'

    def initialize(view_paths = [], assigns_for_first_render = {}, controller = nil)
      view_paths = Rails.configuration.view_path
      super
    end

    def render_contract(contract, format = :html)
      @contract = contract
      prev_format = @template_format
      @template_format = format
      output = render(:partial => 'contracts/show')
      @template_format = prev_format
      output
    end
  end

  def create_pdf
    self.contract_pdf = PDF::PDFGenerator.render(ViewRenderer.new.render_contract(self))
  end
end

Association type cast or: Building instances of a given class on polymorphic/STI collections

In quite some cases you want to build items on a rails collection instead of appending them, especially when the parent object is already stored in the database and you do not want the new item being saved automatically. However, this approach does not work on polymorphic collections as you cannot specify the class of the item being built in the build call. You can find some solutions on the net interpreting the type attribute, however I consider this unsafe because attributes tend to be taken directly from the params hash (actually, type attributes are filtered by default on mass assignments which does certainly make sense). The following code snippet adds a build_a method to has_many collections which allows you to say

collection.build_a SomeModel, attributes

without changing the behaviour of the original build method.

class ActiveRecord::Associations::AssociationCollection

  def build_a(klass, attributes = {}, &block)
    if attributes.is_a?(Array)
      attributes.collect { |attr| build_a(klass, attr, &block) }
    else
      build_record_with_type_cast(klass, attributes) do |record|
        block.call(record) if block_given?
        set_belongs_to_association_for(record)
      end
    end
  end

  def build_record_with_type_cast(klass, attrs)
    attrs.update(@reflection.options[:conditions]) if @reflection.options[:conditions].is_a?(Hash)
    record = @reflection.build_association_with_type_cast(klass, attrs)
    if block_given?
      add_record_to_target_with_callbacks(record) { |*block_args| yield(*block_args) }
    else
      add_record_to_target_with_callbacks(record)
    end
  end

end

class ActiveRecord::Reflection::AssociationReflection

  def build_association_with_type_cast(klass, *options)
    if klass <= self.klass
      klass.new(*options)
    else
      raise ArgumentError, "Expected #{self.klass} or subclass of #{self.klass}, got #{klass}"
    end
  end

end

Logging your model with a custom log

Sometimes it can be handy to keep track what a model is doing in a separate log. Here’s an easy way to do it.

class Article < ActiveRecord::Base

  LOGFILE = File.join(RAILS_ROOT, '/log/', "article_#{RAILS_ENV}.log")

  def validate
    log "was validated!"
  end 

  def log(*args)
    args.size == 1 ? (message = args; severity = :info) : (severity, message = args)
    Article.logger severity, "Article##{self.id}: #{message}"
  end

  def self.logger(severity = nil, message = nil)
    @article_logger ||= Article.open_log
    if !severity.nil? && !message.nil? && @article_logger.respond_to?(severity)
      @article_logger.send severity, "[#{Time.now.to_s(:db)}] [#{severity.to_s.capitalize}] #{message}\n"
    end
    message or @article_logger
  end

  def self.open_log
    ActiveSupport::BufferedLogger.new(LOGFILE)
  end

end

Extracting symbols or keys with TextMate

The times I’d like to get a bunch of symbols from other parts of code, like say, a migration, are plenty. Finally took some time to scribble down a script to do that.

#!/usr/bin/env ruby
EXCLUDED_WORDS = ["option", "options", "default"]
selected_text = STDIN.read
words = selected_text.gsub(',','').split(' ').each {|s| s.strip!}.select{|s| s.include?(":") }.reject{|s| EXCLUDED_WORDS.include?(s.gsub(':','')) }
# Rails console output attributes for a model with the colon in the end of the name, so let's put it in front
words.each do |word|
  if  word.reverse[0].chr == ':'
    word.chop!
    word.insert(0, ':')
  end
end
print words.join(", ")

Make sure you set the output to create new document otherwise you’ll replace your current code. I’d love to see TextMate have the feature to output straight into pastebin.

Sorting arrays and other things in TextMate

You also have these stacked arrays or hashes in your code? Like when you have a pretty fat model with long attr_accessible statements?

I’ve always been bugged they tend to take some time to eye through when they’re not sorted or just on a very long line. And sorting them by hand sure can take it’s time.

I scribbled together a TextMate command to do that for you.

Open up the BundleEditor in Textmate, make a new command and paste following into it:

#!/usr/bin/env ruby
selected_text = STDIN.read
text = selected_text.gsub(/\]|\[|\}|\{|\n|\t/,'').split(',').each {|s| s.strip!}.reject{|s| s.size.zero? }.sort
make_split = text.size >= 6 ? true : false
text = text.join(", ")
['{','['].each {|c| text.insert(0,c) if selected_text.include? c }
['}',']'].each {|c| text += c if selected_text.include? c }
unless make_split
  print text
else
  commas = 0
  intendation = ENV['TM_LINE_INDEX'].to_i - selected_text.length - 1
  text.each_byte do |b|
    print b.chr
    commas += 1 if b.chr == ","
    if commas >= 3
      print "\n"
      intendation.times { print " " }
      commas = 0
    end
  end
end

Now, assign it a hotkey, and following settings:

The code splits on commas, sort each element, line break and auto-intendent according your selection.

Greeting all media peers from Ireland…

…where you can yield, even if no block is given :)

giving &it a shot

Ruby 1.9 has Enumerable#reduce, while we’ve got Symbol#to_proc … which enables us to give enumerables some nifty method injections, right between the eyes, erm, elements:

A Means To An End

Most regrettably, Firefox on Mac OS X ignores your DefaultKeyBinding.dict file if you installed one to get Home and End keys working like you’re used to. Here Matt Ball comes to the rescue. With Firefox Keyfixer for FF3, you can finally forget Ctrl-A and Ctrl-E!