Miscelaneous Debris

fREW Schmidt

2015-03-19


Miscelaneous Debris

fREW Schmidt

2015-03-19



Outline



  • Docker

  • DBIx::Class

  • DBIx::Class::Helpers

  • drinkup

  • Dogma

    • Simplicity, Async



Docker: Agenda



  • What even is this thing
  • Brief History
  • In the small
  • In the large
  • Misc
  • Example / Demo



What even is this


  • Process grouping
  • Process isolation
  • sorta like crappy virtualization
  • docker makes this accessible and even fun



History



  • chroot


  • cgroups

  • LXC




In the small



  • Dockerfiles

  • images

    • layered


  • docker {build,run,stop,rm,rmi,images,ps}

  • awesome for out-of-the-box dev
    • (like databases or other deps like that)

  • haven't had much luck with own code in dev
    • -f probably helps

  • no silver bullet for repeatability

    • esp when initially containerizing

      • carton



In the "large"




  • docker push mapapp.registry.my.domain.com:5000

  • docker pull mapapp.registry.my.domain.com:5000

  • run.sh (sortav a bummer)

    • (this is where the moving parts are)



Misc



  • pause/ unpause - I use this to save battery

  • sysdig - great observability tool w/ container support

  • linking containers

    • complicated but powerful, "safe", and composible

    • I've only used it for network access

  • testing

    • ironically, this can be frustratingly hard

    • unless you already have CI


  • each needed port and volume is a moving part

  • basically means that image is "incomplete"



DEMOS




  • build it!

  • run it!

  • sysdig it!

  • tail logs!


DBIC



  • This is the smallest of overviews

  • I'm sorta bored of doing DBIC talks

  • ... but feel free to ask questions afterwards!



DBIC parts


  • Schema

    • set of tables

  • ResultSource

    • table (go away "actually" guy)

  • ResultSet

    • query

  • Result

    • Row



Relationship Types



  • has_many

    • other rows have references to my PK

  • belongs_to

    • I have a references to some other PK

  • has_one

    • single special case of has_many

  • might_have

    • zero/one special case of has_many

  • many_to_many

    • for join tables (not a real rel)



THAT'S ENOUGH


QUESTIONS





DBIx::Class::Helpers



  • A bunch of handy little components I made for DBIC

  • This talk will go over some of my favorites

  • There may be more useful ones you should look at, so
take a peek



::ResultSet::CorrelateRelationship


    $cd_rs
       ->correlate('tracks')
       ->related_resultset('plays')
       ->count_rs


  • Can be used in lots of places like order_by or columns

  • See especially later slide, ::Row::ProxyResultSetMethod



::ResultSet::DateMethods1


    $rs->search(undef, {
       columns => {
          count => '*',
          year  => $rs->dt_SQL_pluck({ -ident => '.start' }, 'year'),
          month => $rs->dt_SQL_pluck({ -ident => '.start' }, 'month'),
       },
       group_by => [
          $rs->dt_SQL_pluck({ -ident => '.start' }, 'year'),
          $rs->dt_SQL_pluck({ -ident => '.start' }, 'month'),
       ],
    });


  • Simplifies date math for your brain (before/after vs *<* / *>*)

  • Converts DT objects for your DB

  • Extracts Date Parts

  • Adds dates together

  • No subtraction yet (super varied, dubious usefulness)



::ResultSet::IgnoreWantarray


      my @values = paginate($rs->search(...))

  • Fixes that bug



::ResultSet::SearchOr


    sub not_passed ($self) {
      $self->search_or([$self->failed, $self->untested])
    }

  • Sortav a ghetto union



::ResultSet::SetOperations


    $rs->union($rs2)
       ->intersect($rs3)
       ->except($rs4)

  • Not a ghetto union

  • The weirdest feature not to be in core



DBIx::Class::Candy


    package MyApp::Schema::Result::Artist;
 
    use DBIx::Class::Candy -autotable => v1;
 
    primary_column id => {
      data_type => 'int',
      is_auto_increment => 1,
    };
 
    column name => {
      data_type => 'varchar',
      size => 25,
      is_nullable => 1,
    };
 
    has_many albums => 'A::Schema::Result::Album', 'artist_id';
 
    1;

  • Great for slides!

  • (All the following slides assume it)



::Row::OnColumnChange


    before_column_change amount => {
       method   => 'bank_transfer',
       txn_wrap => 1,
    };
 
    sub bank_transfer ($self, $old, $new) {
      my $delta = abs($old - $new);
      if ($old < $new) {
         Bank->subtract($delta)
      } else {
         Bank->add($delta)
      }
    }

  • Just a really useful generic helper ~~ a trigger



::Row::ProxyResultSet{Method,Update}


*RS Method*


    sub with_friend_count {
       shift->search(undef, {
          '+columns' => {
             friend_count =>
                $self->correlate('friends')
                     ->count_rs
                     ->as_query
          },
       })
    }


*In Row*


    proxy_resultset_method 'friend_count';

  • Access already selected data

  • Or fetch if it wasn't, and cache it!

    • (Considering making this warn)



QUESTIONS?


(you can ask about DBIx::Class::DeploymentHandler if you want)


drinkup



  • Just a thing I made a while ago that's fun to talk about

  • Was recently asked about cool stuff I'd made

  • Thankful that one of those things is OSS

  • List of cool things in no particular order



Cool Thing α



  • Cute URLs:

/drinks/
/drinks/data/1
/drinks/find_by_name?name=Mojito
/drinks/search_by_name?name=*tini
/drinks/with_some_ingredients?ingredient_id=1&ingredient_id=2
/drinks/with_only_ingredients?ingredient_id=1
/drinks/without_ingredients?ingredient_id=1


  • Pretty old hat these days I guess...



Cool Thing β



  • Nice web backend



    • Vaguely the inspiration for Tim's WebAPI::DBIC, (which does much more I promise)



Cool Thing ɣ



  • Ingredients are all automatically interchangeable

  • Matpath!

  • Too bad taxonomies are the worst



Cool Thing δ



  • Efficient set based searching
https://github.com/frioux/drinkup/blob/master/lib/DU/Schema/ResultSet/Drink.pm#L164

  • Fundamentally based on:
    • What drinks can be made from the ingredients in a given RS
      • Plus a range of other ingredients

  • eg if I want to go to the store but only buy a few things:
    • what drinks can I make if I am ok with buying 1 to 3 ingredients
   $drinks->ineq($inventory_rs, 1, 3)


µDogma: Agenda



  • Simplicity

  • Async

    • linked, not sure why



Simplicity



  • DBIx::Class is great
  • Catalyst is great
  • ExtJS is great

  • But sometimes, they are all overkill



Simplicity



  • project runs only one DB query → DBIx::Class is overkill

  • project with only 5 endpoints → Catalyst is overkill

  • etc



Simplicity



  • Big frameworks can make you feel more productive

  • ... while making you less productive

  • ... don't use them for everything!



Simplicity



  • Hard to define

    • Examples: Proof, LNM

  • µServices can help here

    • (forking (probably) helps RAM usage)



Right tool for the job



  • Vanilla DBI (or maybe at least DBIx::Connector) is fine!

  • Small web frameworks are fine!

  • Again, µServices can be of benefit here

    • ... use what works in each context



Async



  • async is simpler than polling

    • (I think?)

  • async can definitely add complexity

    • Hide it with OO!

  • Async requires a lot of mental reconfiguration

    • ... but that reconfiguration more accurately matches the real world



Async vs Sync



  • Sync

    • writer → DB → while(1)


    • Problems:
      • sleep 1

      • Write contention


  • Async

    • writer → MQ → reader


    • SOLUTIONS!
      • Look ma, no sleep!

      • MQ's tend to be able to handle heavy writes better



Futures



  • Callbacks

sub {
    ...
    sub {
         ...
            sub { ... }
       }
    }


  • Futures

my $f = $http->do_request(...)
     ->then(sub { ... })
     ->then(sub { ... })
     ->then(sub { ... })




Futures 2



  • Callbacks are the MOST BASIC and will likely need to be around forever

    • everything else is an abstraction of these

  • Futures

    • represent a single outstanding action from a single direction

    • you have to store them

  • There are other abstractions that barely exist in perl




The End