hello everyone welcome to my channel my
name is evident and in today's video I
am going to show you how to upload large
files to github so if you've ever used
to get help to upload files you know
that when you upload using the github
user interface you can only upload to a
fat megabyte at a time
so if I go to this data set so from here
if I go to add file upload files if I
try to upload files from this system and
can only upload up to 25 megabytes at a
time if I upload files using the command
line get add git commit and get pushes
in the command line
I can only upload 100 megabytes of data
so if I had a CSV file that sees 100
megabytes or less I could upload it
using the command line and if the file
is to in fact make up bytes or less I
cannot upload it using the github
interface but what if you have a file
that is more than 100 megabytes
naturally you can't upload more than 100
megabytes to github for you to upload
more than 100 megabytes at github you
have to use something called github
large file system and in this video I'm
going to show you how to use it so if I
go back to this data set this data set
right here is 289 megabyte so how did I
get tuna in a megabytes of data to
github I'll show you that so it's
actually very simple you just go do
something called github LFS just google
github LFS
and this will pop ask github large file
storage system and from here you click
on download you have to download this
first so I'm also going to leave this
link in the description of this video
but the first step is to download this
gets large file system and after you
download it you run it alright after
downloading this and then running it the
next step is your command line so then
you go to your command line and it
doesn't really matter where you do this
but just type in git LFS install and
then you get this I already have it
installed so just give me a bit that
gets hooks and git LFS initialized
that's the first step and all the
instructions is on this page to search
it's not really rocket science
and of course I'm going to share this
link in the description below so after
you do creat LFS install the next step
is to add the files that you want
tracked alright so after you have
downloaded get large file system and
after you have installed it and after
you've run this command in your terminal
and you have to run this command only
ones like after you've done the first
time it have to do it over ever again
after that is done you are pretty much
done setting up the basics now whenever
you are working on a project and
whenever you want to add stuff to github
you just navigate to the folder that has
the file you are trying to upload and
you run this command so right here you
found that to add these two gates up so
right
this folder is not actually a pastor
github but if I wanted to add this
github like if I wanted to add CSV file
to github a lot to do it you'd get LFS
track and here I'll do dot CSV and if I
run this command is going to
automatically track these years we found
that's about 289 megabyte now I'm not
actually going to run this command
because I don't have this photo attached
to github right now I'm just using this
as an example and in the future if you
want to track and difference by phone to
track audio files it could be an mp4 I
mean mp4 MPEG mp4 or it could be
something like an audio file so it could
be dot with for example if you have a
waveform to make up I'd like I don't
know why you have audio furnace over 100
megabytes or if is if I could see so
whatever found that you want to add to
github that is over 100 megabytes you
just add this here and make sure you are
doing this on the folder and under the
directory that is attached to github and
the director you're working on so what
so just like you do get argot commit on
the directory that is attached to github
on the director you are tracking make
sure that every time I walk in on your
project you have to do this okay and
once you run this command it is going to
add whatever file that is in here to get
hub see you don't even have to do and
this you can specify the name of the
file if you wanted to after you run this
command in S
thing is to make sure you add your get
attributes so they just type in the
command get add get attributes or you
can just copy and paste this and after
you type in this command you can just
click enter and that's pretty much it
you are done you can move on with your
project and then you can do get add get
commit and get push as usual and
whatever file that you have that is over
100 megabytes that you've added using
git LFS track will be uploaded to github
and that's basically how I was able to
get this that I said that's over 100
megabyte unto github
now that you've learned an easier to add
file everyone de megabytes at github
I'm going to have to warn you about
something you have to be I don't know
how to describe it careful I guess so
basically this is free like whenever you
are adding files so under makeup bite so
github is free up to a certain limit and
I didn't know this I found it about it
the hard way
ok so basically with github LFS system
you get 1 gigabytes of storage and so
far is 0.5 out of my 1 gigabyte of
storage and you get one gigabyte or
month of bandwidth this basically means
that if you run a command like pandas
with CSV and you are reading a CSV file
that is uploaded using the github LFS
system that count against your bandwidth
and you only get about one gigabyte of
bandwidth per month and if you want to
use more than one gigabyte of bandwidth
per month you have to pay github $5 a
month
I mean $5 a month you get 50 gigabytes
of bandwidth per month and you get 50
gigabytes of storage per month so that
something had to be aware of like I
didn't know this until I was working on
your project you know the happy news
project and I kept I'm running my CSV
file I'm like a man I got email from
github saying that hey you've used too
much bandwidth this month and you have
to pay us for you to keep using github
LFS system and I didn't pay them
anything I didn't put in my card
information and then I got another email
that says my github LFS system has been
suspended for this month because I used
too much bandwidth I use more than my
one gigabyte per month bandwidth so just
be aware that yes github LFS system is
free but there are limits I want to go
over that limit you have to pick its hub
$5 per month if you don't pay them $5
per month or suspend your github LFS
system LFS storage access basically and
I don't know what the suspension really
means maybe it means I can't upload more
files or it means I can't assess a file
that's using the github LFS I don't know
what the suspension means but before I
got I'm suspended for the month I was
able to finish doing the work that I
needed to do using that data set so I'm
going to add this file to the comments
section of this video
and just a recap you download it and you
run the file that you downloaded then
you go to your command line it doesn't
matter your command line you run this
then after you run that you whenever you
are walking on it github project
whenever I work on any project I need to
upload father's over 100 megabytes you
run this on the folder on the directory
that's you are working on under director
that's your put into github you have to
do this every time for every new project
and then you have to do this every time
for every new project and this is only
once at the beginning and when you
project and then you can do get I'll get
upload get commit and get upload you
have to do it you can do get advocate
comments get push as usual and just be
aware that this is your storage limit
and this is your bandwidth limit unless
you want to take it it gets half five
the last a month so have 50 gigabytes of
storage and 50 gigabytes of penguin this
is the best way I have found to upload
large files online and be able to access
it for there a science project I know
before in the past when I tried to work
with large files
I'll be angry and frustrated that I
wasn't an easy straightforward way to
upload large files online but this
github large file systems one of the
best ways to upload large files online
but there are limits unless you want to
pay them five dollars a month all right
that's basically it for this video
remember you can always visit me online
at evidence and dot-com this is my
website and this is where you read on my
blog post and also if you go to mission
learning education calm you can get
access to my free data science resources
and any resources that are using my
youtube videos i'm and blog post can be
found right here that's it for this
video thank you for watching and I'll
talk to you on the next one bye