2022 Installing TensorFlow Keras & Python 3.9 in Mac OSX M1

Hi this is Jeff Heaton, welcome to Applications of Deep Neural Networks, with Washington University.In this post I am going to show you how to setup a Mac In particular a newer Mac, one of the M1s.That have the Apple Silicone, rather than Intel.withKeras, Tensorflow, everything you need for the course in Deep Learning.If you have a Windows computer, or an older Intel Mac, I have other posts on those which are linked in the description.And I suggest you take a look at one of those.There is a few steps to this, it is really not that bad.And I do want to explain some of the differences between the MiniForge and Miniconda versions that you will encounter with this.Also, if you do not wish to install the actual software onto your computer, you can make it through the entire course with Google CoLab.If you've watched some of my other posts you know that I use Anaconda and Miniconda quite a bit.These are two Python platforms, in particular, both made by the same group That you can use on just about any platform, be it Windows, Mac, or Linux.In this post I am going to show you how to use Miniforge, and Miniforge is something new, at least to me.Miniforge is supported on a variety of platforms as well, but I've started using it primarily on the Apple M1.I suggest trying to get by entirely with Miniforge and not necessarily even installing Anaconda.Because you will need both of them installed if you do want to make use of Anaconda.Because if you are using Anaconda, you are not necessarily, or really, at all, be able to access the GPU acceleration that Apple has built into this computer.So, on this machine I am going to show you how to install Miniforge.It will just be Miniforge,without Anaconda.I do actually run both of them on my Mac, I might do another post on that, if you are interested in how to run them side-by-side.Definitely let me know!But the point isMiniforge, if you look at the reason they have it, "An emphasis on supporting various CPU architectures" including Apple M1.Personally, it annoys me a little but that I can't run everything through Miniconda, and I have to have two entire Python instances on my mac, but, thats the way they want to do it.so, lets go to my class's website.There's various instructions on how to do this.I going totake you through the process that I've, somewhat developed, based on others, this is a little bit different than some of the ones you will see here, but I think its pretty streamlined and it gets you everything that you will need for my course installed.If you go into T81 Deep Learning, and you go into install, there is a variety of files, because, for one thing, they keep updating these various platforms, and updating the install instructions, and the instructions for installing my course has changed a number of times since I first put this out here.and I don't like to put broken links onto older YouTube posts, so they link to their respective version.This post is going to make use of the: tensorflow-install-mac-metal-jul-2021.ipynb And if I make some changes to the file, that do not break the post bad enough that I need to rerecord it, then I will put a linkto the newer version here, I will probably, if they are not breaking changes, I will probably modify the file in place.So these are the instructions.I talk about basically I just told you about, in terms of dealing with the multiple versions, and dealing with Miniforge.I suggest staring by installing Miniforge.I like to use Homebrew to do this.I am going to show you how to install Homebrew, its similar to yum or apt-get if your used to Linux.It is kind of like the command line version of the Apple Store, if you want to think of it, for that.If you go to the Homebrew page, This command is the installation, that is all it takes. So we copy that.And I am going to open up a terminal.I actually do not have,Homebrew installed on this Mac, yet.Because I just got this mac, and I put my stickers on it, so I guess its mine now.Not gonna send it back!Lets go ahead and paste.That into there.And go ahead and press [ENTER], and its going to run that.So it wants to run as "su do," obviously, it wants to run as root.So, I will go ahead and enter my password.And, there is that.It is also going to install the XCode command line tools.that, you need thatfor so many things on a Mac.Very useful to have, I don't... if you do some of the other more manual processes for installing this, You will have to install that as a separate step. So I am going to press [ENTER].And now, its installing Homebrew.We will go ahead and fast forward through this.By the way, I will probably need to put some more stickers on this laptop, any suggestions?Let me know in the comments.While I am continuing to wait for this.Let me also discuss a few other considerations for this.The Apple Metal, which is the Apple equivalent, almost, of NVIDIA CUDA.Will not run everything in my course. I would say it will run a solid 90% of it.But, some of the advanced code that I have, say GANs, Makes use of two things, in particular. And one is PyTorch.Currently is alot more difficult to get working on M1.There is currently issues open, that have not been resolved, yet.Hopefully, that will resolve soon.But the StyleGAN2 ADA, that we make use of in this course, Makes use of actual CUDA code, which is actual C99 code, C99 type code.That actually defines custom kernels for this, so, Obviously, that is not at all compatible with Apple Metal.You could certainly write this in Apple Metal, but, StyleGAN2 ADA was not written that way.So, more advanced machine learning, where it is custom kernels, the Mac M1 is great, butthe Apple Metal is great, but its just not as big of an ecosystem as NVIDIA CUDA.So, it's not going to see as much compatibility.Now, everything you will need for the course is available in CoLab.So, if you're going to run it on your Mac, like I said a solid 90-95% will run just fine.But some of this just will not work.Would you be interested in a post that shows what works and what does not on an Apple M1?At least, in my experiance.Let me know in the comments. Also is this post helping you to install? Please give it a like, thank you.Lets continue with the fast forward.I really dislike processes that make you enter the same password.It prevents you from letting them run without you watching every step.Okay, its done with all of that.This is an important step, you're supposed to add HomeBrew, to your path, and by the way, this assumes that you are using Z-Shell.Otherwise some of this can be different.Mac used to use the Bash Shell, but currently they are using the Z-Shell.Unless you specifically changed that, it will be the case.We will go ahead and run that, okay, its modified that.Then we will run that.That just reloads the shell.I will go ahead, though, and exit the terminal.And start up a new terminal, just, Just because I do not completely trust that everything gets completely reconfigured like it should.Now, I should have the "brew" command from the command line; and I do.So, this is alot like "yum" or "apt-get" or those kinds of things.Lets go back to my instructions.So we installed HomeBrew, we followed those instructions.They also recommend installing this, I think it was basically done by Brew, but I will go ahead and execute that.Already installed, so we are good there.And then we are going to do, "brew install miniforge".There's a variety of ways to do this. You can also download a shell command.from, you. can see it in this install, so if this isn't working for you, you may want to try that.as well, but I am going to go ahead and copy this.Run it from here.So its going to make use of Brew, and its going to install Miniforge.The only way I can think of this failing is, If there is a new version of Apple Metal, and a new version of Miniforge, and its not compatible, if I run into that I will defiantly post, something about that, then you would request a very specific version of Miniforge.We will fast forward though all of this. The distinctive (base).That shows you what environment you're in, currently, we're in the (base) Python environment.If I do "which python",We'll verify that.So see we're the Python out of Miniforge 3, then "python".So next step.We're going to install Jupyter.Jupyter is sort of the IDE that you will use to execute all of the code in this course.You can also use Jupyter Lab.Or PyCharm.We will go ahead and do this, this will take, ah, its already done.So, that's handy.Now, what I am going to have you do.Normally there are several things you have to install. You have to install TensorFlow.And then the Metal plugin.I also have a handful of machine learning packages that I like to install for my course. They are all very general, so it would not hurt to install them.Even if you're not taking my course. You can see them all here. Its Scikit-Learn, Pandas, all very standard stuff.You will need this Apple TensorFlow YAML file.And you will execute this command.I already have this loaded on to my system.So I will go into where I have it.Because I did a clone on my class, and you can see the YAML file here.If you do not want all of the material from my course just download the YAML file.I have a link there, just make sure you are in the same directory as where you downloaded when you run the command.You will need that YAML file.That I just mentioned.If you do not want all my course material, just download the YAML file. I have my course material already loaded.And there it is.I am going to go ahead and execute this conda environment create command.It is going to go ahead and, create a "tensorflow" environment that has all of those packages installed, as well as TensorFlow and the Metal plugin, that is going to let you use the GPU capabilities of your M1 Mac.We will fast forward through this, it tends to take a bit of time.So now, we will do the next step, which is "conda activate tensorflow", which is in my instructions.And there we go, we are now inside Tensorflow.And I am going to install "nb_conda", which lets me, Lets me link this Tensorflow kernel I've created, into Jupyter, and it does all of this.Procede? Yes! And we will fast forward.Alright, I will show you what that is in a moment.And we have to do this command here, which is going to actually link it.you can have multiple of these installed in your, Jupyter, and that is what I very commonly do for different projects that I am working on. I don't try to create one...Python environment that has everything that I need, because that gets unwieldy and I lose track, I tend to create an environment for each.or often a Docker image for each, depending on what I am doing.All right, we will enter that.It's now linked into Jupyter.So now we will go ahead and run Jupyter notebook, which is the next step.In my instructions.And here we are in Jupyter notebook.Going to create a new notebook, or I could just load, the file we were running from, if you downloaded it.But I will just go ahead and create a new...environment of TensorFlow 3.9.that is the environment we just created.Very important that you select that or you may not have TensorFlow installed.So we will go back to here and copy and paste this code that will let you know if...If everything worked, this is where you cross your fingers.I will go ahead and run this, it takes a minute, fast forward.You can see these three coming up here, which is part of Apple Metal, which is a good sign.Saying GPU available is an even better sign. So we are completely installed and ready to go.And if you want to check it, just load up any...code that uses TensorFlow, just like you normally would. Any of the like training the ResNet would be a example in...my material. Go ahead and run it.You'll see it start up.Metal device set to Apple M1, all very good signs.And if you go to the activity monitor, and see the percent GPU,will, there it is, it takes it a moment to get started, so if you don't see something on the, percent GPU at first, don't panic.it may also be downloadingSo the fact that we are on epoch 2 and the GPU is pegged at 88%, this is actually a good sing, so, if you're seeing thiscorrectly, congratulations, you have the GPU working.Thank you for watching this post, please subscribe to my blog and follow along with the course.Or if you are one of my students, welcome to the course, we will have alot of fun learning deep learning.Thank you for watching the post, and if it was helpful, please give the post a like.