馃攳
Designing a serverless finance system on Google Cloud - YouTube
Channel: Google Cloud Tech
[0]
hi dina
[2]
hi martin how's it going
[4]
going great because today we are
[6]
designing an application that deals with
[8]
money i hear oh oh yeah that's true
[12]
for our event planners
[14]
right uh so it's gonna be serverless
[16]
application that deals with money we're
[18]
gonna go through exactly what products
[20]
you use in this thing are you ready i am
[24]
let's do it
[25]
[Music]
[35]
okay
[36]
so we have
[38]
three users
[40]
we have our event planners our
[42]
accountants and our vendors and our
[44]
event planners need to
[46]
register the vendors
[48]
they need to create purchase orders so
[50]
that the vendors deliver whatever it is
[52]
that they're delivering
[54]
the accountants need to view and improve
[57]
invoices so the vendors get paid and the
[60]
vendors need to be able to submit their
[62]
invoices
[64]
very good so i see we have them here on
[65]
the left hand side looking good and
[68]
let's see some of these are inside our
[70]
company some are outside huh yes exactly
[72]
we can think of this as like two groups
[74]
like one's like internal and one is
[75]
external
[76]
got it and the internal ones that's it
[79]
looks like event planners and
[81]
accountants
[82]
yeah exactly so
[84]
we're you know we're going to build a
[85]
web app um
[87]
but yeah it's worth starting by thinking
[89]
how are we going to ensure
[91]
that only the people inside our company
[94]
access this web app
[96]
ah right so for a project i recently was
[100]
on we used identity aware proxy for that
[102]
it is super simple to set things up that
[104]
people within our domain they get in
[106]
nobody everybody else is kept outside
[109]
and you can even set levels within that
[111]
but but yeah for this super easy let's
[113]
do it identity aware proxy awesome okay
[116]
so the next question is
[119]
what do we use for the web app
[122]
yeah
[123]
right it's a web app okay serving html
[126]
javascript and so on uh yeah i think app
[128]
engine would work well here you and i
[131]
both know app engine well um it's also
[134]
uh on again on my last project it was
[137]
like a two or three click integration to
[139]
put it together with identity web proxy
[142]
even better and super classic love it
[144]
okay
[145]
this is easy
[147]
so we're done
[149]
no okay
[151]
next we come to
[153]
possibly the first difficult decision um
[155]
we have to start thinking about storage
[158]
because the event planners are going to
[160]
be you know
[161]
filling out a form and saying these are
[163]
the vendors and that needs to go
[166]
somewhere
[168]
and
[169]
i i like to think about storage options
[172]
always as
[174]
relational databases versus
[176]
non-relational databases
[179]
and in this case you know we're talking
[181]
about
[183]
um i'm talking about money and we're
[184]
talking about
[186]
these are the purchase orders and then
[187]
these are the invoices and we're going
[189]
to need to be able to
[191]
to run reports against these things say
[193]
like you know this is like the money
[194]
that we've pledged this is the money
[195]
that we've paid
[197]
and so for something like that i really
[199]
think that we absolutely have to use a
[202]
relational database
[204]
yeah i agree uh
[206]
for like you're saying sum up uh like a
[209]
thousand documents a thousand invoices
[211]
and get their sum super easy and quick
[213]
to do uh in and i think uh we could do
[216]
spanner we could do various other things
[218]
i think cloud sql is the right here
[221]
because we will have on the order of you
[222]
know
[223]
gigabytes not terabytes or petabytes
[226]
here so cloud sql which is well known
[228]
you and i both familiar with mysql and
[230]
postgres so that'll be easy oh my gosh i
[233]
feel like it's 2010 app engine with a
[236]
cloud sql database i love it
[238]
yeah well-known technology
[241]
okay cool so
[243]
uh let's see everybody's go oh no
[246]
vendors aren't connected to anything
[248]
yes okay so the first thing that we need
[251]
to think about is
[253]
is the fact that they are external and
[256]
they're coming from anywhere and
[257]
possibly
[259]
possibly everywhere um so how do we
[262]
ensure that they are able to access
[266]
our service
[267]
and do appropriate things on the service
[269]
but nobody else is
[272]
right so we need to know that they are
[274]
who they say they are and and so on i
[276]
think for that identity platform is a
[279]
great choice because then they can log
[281]
in with their gmail accounts or from
[283]
various other identity providers they
[285]
can log in
[287]
self-service set up their own accounts
[289]
but we do know who is who if somebody
[292]
logs in as fluffybunny at gmail.com we
[295]
know that that really is fluffy bunny
[297]
the vendor or
[299]
sonomawinery.com maybe
[301]
more likely
[303]
yes
[305]
so identity platform there i think that
[307]
that's a shoe and that would be easy to
[309]
do but
[310]
now what platform serves up html and
[313]
javascript and css to them yeah i think
[316]
let's go with app engine again um you
[319]
know we don't
[320]
i don't think that we need to do cloud
[322]
run for this because
[324]
you know we're not using any special
[325]
runtimes or or you know anything
[330]
that like that i think just your
[332]
standard app engine would work great
[335]
yep i agree i love cloud run but if it's
[338]
smart to keep it to only one product
[341]
here for computing
[342]
uh okay so these folks upload uh
[345]
invoices it says here submit invoice pdf
[348]
is that what the vendors are doing
[350]
oh
[351]
yes okay so
[353]
um
[355]
we always need to consider that we need
[357]
to meet our users where they are
[359]
and in this case our event planners work
[362]
with vendors from all over the place
[364]
sometimes it's like a giant hotel chain
[367]
um that has like very standardized
[369]
systems that can plug and play really
[371]
easily with whatever we want
[373]
and then sometimes it's you know mom and
[376]
pop from a sonoma vineyard and they're
[379]
gonna
[380]
maybe send their invoice on the back of
[383]
a postcard i don't know
[386]
or they even fax them sometimes so yeah
[391]
we need a system that
[393]
allows the vendors to to keep using
[395]
whatever system works for them um
[398]
maybe it's a pdf um maybe they you know
[401]
have like a hand written invoice and you
[404]
know it'd be really cool if they could
[405]
just like you know take a picture and
[406]
upload it yeah with their phone right
[410]
yeah yes yes exactly
[412]
um which means we're back to the storage
[414]
question
[416]
uh
[417]
but this time
[419]
you know it's not
[421]
columns and rows it's not numbers it's
[424]
it's bytes you know it's um
[427]
it's rather
[429]
yeah binary files right yeah and and i
[432]
mean we could theoretically store them
[434]
in a relational database
[436]
but they are products that are better at
[438]
it right yes yeah and you know
[441]
we don't want to store them
[443]
you know for too long either um so
[446]
i was thinking that maybe we could use
[448]
something like storage bucket a cloud
[450]
storage bucket
[452]
i love that google cloud storage because
[453]
you said we don't want to keep them
[454]
around for too long
[456]
on a previous project i worked we set
[459]
things up so after six months you can do
[460]
this very easily with a few clicks in
[462]
cloud storage after six months
[464]
uh delete the files
[467]
so it's nice and clean everything's
[469]
automated you have automated controls
[471]
yes this is good
[472]
yes and you can set like kind of like
[474]
granular permissions per storage bucket
[476]
so i think using cloud storage is really
[478]
great
[480]
are we done all right so the picture the
[482]
vendor took a picture with a cell phone
[484]
of the postcard that written their uh
[486]
invoice on they send it up it's now
[488]
sitting in a storage bucket in google
[490]
cloud storage now what
[493]
we need to get this data
[496]
from the bucket
[498]
somehow
[500]
into cloud sql um
[504]
so we we don't want to have a system
[507]
where somebody's got to like log in and
[510]
look at all these images and then like
[512]
you know
[513]
input it
[514]
into a form because that's
[517]
well what's going to happen is like it's
[518]
just going gonna
[519]
take for like take forever like it's
[521]
gonna sit there for a really long time
[523]
people are gonna make mistakes
[524]
um and then the vendors are gonna get
[527]
really angry when their payments are
[529]
super late so right
[531]
i think we need to do some kind of event
[534]
driven development here so that as soon
[537]
as the image goes into the storage
[539]
bucket
[541]
something happens
[542]
that starts us um on our you know in our
[546]
pipeline
[547]
into the cloud sql
[549]
what do you think okay
[550]
yeah i like that and we can easily set
[552]
up a cloud function to do that so if if
[555]
a new document drops in this bucket then
[557]
trigger this cloud function and the
[559]
cloud function gets the
[561]
the document
[563]
so and can work on it okay so the cloud
[565]
function now has access to this
[568]
image taking a pdf or
[571]
image taken with the phone and now we
[573]
need to pull data out of that how do we
[576]
do that
[578]
with machine learning
[581]
of course but
[583]
i like your enthusiasm but i'm not
[585]
really a machine learning expert to dina
[588]
oh my gosh neither am i but the great
[590]
thing is you don't have to be now oh
[594]
we use machine learning not just because
[597]
it's cool because it's a buzzword
[600]
machine learning is is the tool to use
[603]
when you need to make decisions on
[606]
things that you don't know about yet um
[610]
you have seen examples of it in the past
[612]
and you need your system to be able to
[613]
look at it and make a decision without
[616]
having an exact recipe a list of steps
[619]
to follow and so
[622]
this is a perfect example because we
[623]
don't know
[624]
what fields are going to be on this form
[626]
we don't know if they're going to be
[627]
like writing little handwritten messages
[629]
with like hearts on it um we so we need
[632]
some kind of machine learning api to be
[635]
able to go in and parse the information
[638]
now luckily google has a lot of products
[641]
that
[642]
that already have pre-built models for
[645]
you because i do not want to go in and
[647]
you know build my own model for this
[651]
google has probably seen far more
[653]
documents than you or i have seen or can
[656]
come up with yeah exactly exactly and
[658]
you know i don't have the time to to
[660]
collect a million examples to be able to
[662]
train a new api so i think we should use
[665]
um the vision api because the vision api
[668]
has a feature that lets you parse forms
[673]
excellent cool so vision api there we
[677]
run that and then the output of that is
[680]
like
[682]
it can tell things like tax was a
[684]
hundred dollars uh the total was five
[686]
hundred dollars that sort of thing you
[688]
pull lots properties
[689]
exactly exactly so like it'll go through
[691]
and i'll find the labels and then i'll
[692]
find the values and then you have this
[694]
this you know
[695]
structured data um
[697]
but the the only thing is like we need
[699]
to be careful about like trying to put
[700]
this directly into
[703]
something like a relational database
[705]
because if you don't know
[708]
all the fields that are going to be on
[710]
the form in advance
[712]
ah right uh i think i am the just the
[715]
right thing so we'll we we have
[717]
documents right with certain properties
[719]
we don't know exactly what properties
[721]
there is a database product built for
[723]
that uh firestore it's a document
[725]
database so each document can have
[728]
different uh properties from other
[730]
documents as well so it's sort of semi
[733]
it's more
[734]
ordered than just an image but it's not
[737]
as strictly uh ordered as a sql database
[742]
perfect perfect okay so then um
[746]
it it you know we have the event
[748]
triggering the cloud function uh the
[750]
vision api looks at it and says hey you
[752]
know this is the the name of the event
[755]
this is the vendor this is how much it
[756]
cost it puts all that into firestore and
[759]
then when our accountants log on to our
[761]
app engine web app uh we'll pull up that
[764]
document that from firestore and then
[767]
they'll be able to look at it and
[768]
they'll be able to approve it or to be
[770]
able to you know like edit it slightly
[772]
um you know maybe look at the image and
[774]
say like oh no this wasn't for
[776]
um
[777]
mom and pop vineyard this was for
[781]
uh
[782]
sonoma i don't know
[785]
right or one image one number was a
[787]
little smudged uh and then in that one
[790]
out of 100 cases they can go in and
[791]
tweak it yeah i like that and that would
[793]
do that from app engine and you said
[795]
then that feeds into sql right then
[798]
we're done yay i like this it's sort of
[801]
order out of chaos yeah chaotic data
[804]
comes in and gradually gets more ordered
[806]
along the way and finally it's order
[808]
enough for the accountants to issue
[809]
payment yeah yeah no yeah the great
[812]
thing is that there's these two
[813]
different work streams um
[815]
one
[816]
is just your very very classic
[819]
app engine cloud sql and then the other
[821]
really takes advantage of all of these
[824]
like
[825]
new you know event driven and machine
[828]
learning and firestore all these like
[831]
new technologies and ways of coding and
[833]
ways of thinking and then we put them
[835]
together and like you say order out of
[838]
chaos
[840]
thank you everybody for watching if you
[842]
would like to see me or dina design
[844]
another serverless app please let us
[846]
know in the comments or if you have
[848]
questions about this one
[849]
also let us know what other topics you
[851]
would like to see in this video series
[853]
about serverless computing
[855]
until next time
[857]
[Music]
Most Recent Videos:
You can go back to the homepage right here: Homepage