hygorxaraujo https://hygorxaraujo.com/ Writing on data science, machine learning engineering, and agentic workflows by Hygor X. Araújo, a data scientist and systems engineer specializing in production-grade AI solutions. quarto-1.9.37 Sat, 03 May 2025 03:00:00 GMT Installing TensorFlow 2.19 with GPU support on Fedora 42 Hygor X. Araújo https://hygorxaraujo.com/posts/2025-05-03-installing_tensorflow2.19_on_fedora42.html This post aims to be a simple guide on how to install TensorFlow 2.19 with GPU support on Fedora 42.

If you only want to use TensorFlow with GPU support without all the hassle of installing the NVIDIA CUDA Toolkit (NVCC), it is probably easier to just use it with Docker (Docker install guide). This guide is for those who want to install TensorFlow with GPU support on the host system.

Requirements

  • Python 3.9-3.12
  • GCC 13.3
  • CUDA enabled GPU
  • NVIDIA CUDA Toolkit 12.5
  • NVIDIA cuDNN 9.3
  • NVIDIA driver greater or equal to 525.60.13

Steps

1. Setup GCC 13.3 (based on the guide linked in the references)

The NVCC 12.5 requires GCC 13.3, which is not available in the Fedora 42 repositories. To install it, we need to build it from source. The steps are as follows:

  1. Install the build requirements
sudo dnf group install development-tools

sudo dnf install mpfr-devel gmp-devel libmpc-devel \
zlib-devel glibc-devel.i686 glibc-devel isl-devel \
g++ gcc-gnat gcc-gdc libgphobos-static
  1. Get the source code
wget https://ftp.gwdg.de/pub/misc/gcc/releases/gcc-13.3.0/gcc-13.3.0.tar.xz
  1. Extract the source code tar xvf gcc-13.3.0.tar.xz
  2. Configure GCC
# Make a build directory
cd gcc-13.3.0
mkdir build
cd build

# Configure GCC for the build
../configure --enable-bootstrap \
--enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,lto \
--prefix=/usr/local --program-suffix=-13.3 --mandir=/usr/share/man \
--infodir=/usr/share/info --enable-shared --enable-threads=posix \
--enable-checking=release --enable-multilib --with-system-zlib \
--enable-__cxa_atexit --disable-libunwind-exceptions \
--enable-gnu-unique-object --enable-linker-build-id \
--with-gcc-major-version-only --enable-libstdcxx-backtrace \
--with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu \
--enable-plugin --enable-initfini-array --with-isl \
--enable-offload-targets=nvptx-none --enable-offload-defaulted \
--enable-gnu-indirect-function --enable-cet --with-tune=generic \
--with-arch_32=i686 --build=x86_64-redhat-linux \
--with-build-config=bootstrap-lto --enable-link-serialization=1 \
--with-default-libstdcxx-abi=new --with-build-config=bootstrap-lto
  1. Build GCC (this will probably take a long time) make -j<number_of_cores_to_use>
  2. Install GCC sudo make install
  3. Verify the installation gcc-13.3 -v

2. Install the NVIDIA driver

  1. Install the driver: You can follow the RPM Fusion guide in the references to install the NVIDIA driver. Basically, you need to have the RPM Fusion repositories enabled in dnf and install the below packages:
sudo dnf install akmod-nvidia xorg-x11-drv-nvidia-cuda
  1. Reboot your system
  2. Check the modules with:
# loaded modules
lsmod | grep nvidia

# module information
modinfo nvidia

# NVIDIA system management interface
nvidia-smi

Note: you will need to disable secure boot for the akmod-nvidia to work.

3. Install NVIDIA CUDA Toolkit 12.5

  1. Get the installation script from the NVIDIA website. It will be like below:
# Get the script
wget https://developer.download.nvidia.com/compute/cuda/12.5.0/local_installers/cuda_12.5.0_555.42.02_linux.run
  1. Setup the alternative gcc for CUDA
# Create the directory structure that CUDA will use after installation
sudo mkdir -p /usr/local/cuda-12.5/bin

# Create a link for the gcc 13.3 version
sudo ln -s /usr/local/bin/gcc-13.3 /usr/local/cuda-12.5/bin/gcc
  1. Start the installation Since we are not using the default system GCC from Fedora 42, we need to make sure the installation script will find the correct version. We will do that by overwriting some environment variables during execution: CC and PATH.
sudo CC="/usr/local/cuda-12.5/bin/gcc" PATH="/usr/local/cuda-12.5/bin:$PATH" sh cuda_12.5.0_555.42.02_linux.run  --silent --toolkit --no-opengl-libs

This will install the toolkit with the default options, without installing the NVIDIA driver again and accepting the End User License Agreement (EULA).

  1. Update environment variable in your .bashrc or .zshrc:
export PATH=/usr/local/cuda-12.5/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.5/lib64:$LD_LIBRARY_PATH
  1. Check the installation with nvcc --version

4. Install and use TensorFlow 2.19

Using the uv package manager (you can also install it with pip), we can install TensorFlow 2.19 with GPU support. The steps are as follows:

  1. Create a virtual environment uv venv --python 3.12
  2. Install cuDNN with uv add nvidia-cudnn-cu12==9.3.0.75
  3. Install TensorFlow with uv add "tensorflow[and-cuda]==2.19"
  4. Manually link the required libraries with TensorFlow
# Navigate into the main TensorFlow package directory
pushd $(dirname $(python -c 'import tensorflow as tf; print(tf.__file__)'))

# Create symbolic links to the NVIDIA shared libraries (.so files)
# This links all *.so* files from../nvidia/*/lib/ into the current directory
ln -svf ../nvidia/*/lib/*.so* .

# Return to your previous directory
popd

# Create a symbolic link for ptxas (CUDA assembler)
# Find ptxas within the pip-installed nvidia_cuda_nvcc package and link it to the venv's bin
ln -sf $(find $(dirname $(dirname $(python -c "import nvidia.cuda_nvcc; print(nvidia.cuda_nvcc.__file__)"))/bin/) -name ptxas -print -quit) $VIRTUAL_ENV/bin/ptxas
  1. Check the installation with the code below. It should list your GPUs, something like [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
import tensorflow as tf

print(tf.config.list_physical_devices('GPU'))
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

References

Back to top ]]> python tensorflow cuda https://hygorxaraujo.com/posts/2025-05-03-installing_tensorflow2.19_on_fedora42.html Sat, 03 May 2025 03:00:00 GMT Explaining the Fogg Behavior Model Hygor X. Araújo https://hygorxaraujo.com/posts/2020-03-08-explaining-the-fogg-behavior-model.html In this post, I would like to explain the Fogg Behavior Model, created by Brian Jeffrey Fogg (B. J. Fogg).

According to B. J. Fogg’s research, Behavior () happens at the intersection of three things: our Motivation (), our Ability (), and the Prompt () to the behavior.

We can visualize the relationship between motivation, ability and the prompt in a two-dimensional graphic.

I plotted the graphic below using Python and the NumPy and Matplotlib packages. I kept the code for anyone who is interested in how to do it.

Code 
import matplotlib.pyplot as plt
import numpy as np

text = { # text dict for easier translation
    'ability':'Ability',
    'motivation':'Motivation',
    'title':'Fogg Behavior Model',
    'bmap':'B = MAP',
    'bmap_sub':'At the same\nmoment',
    'ability_axis':('Hard to Do', 'Easy to Do'),
    'motivation_axis':('High', 'Low'),
    'action_line':'Action Line',
    'prompts':'Prompts',
    'prompts_succeed':'succeed here',
    'prompts_fail':'fail here',
}

blue_color = '#06abe1'
green_color = '#81a050'
light_green_color = '#d2e5c9'
red_color = '#ff2222'
light_red_color = '#ffcbcb'

# Data for plotting
x = np.arange(0.0, 1.1, 0.01)
y = 1.05 - np.sqrt([np.max((xi, 0)) for xi in 1 - (x - 1.05) ** 2])

fig, ax = plt.subplots()
ax.plot(x, y, color=green_color)
ax.set(xlabel=text['ability'],
       ylabel=text['motivation'],
       title=text['title'])
ax.tick_params(axis='both', which='both',length=0)
ax.fill_between(x, y, color=light_red_color)
ax.fill_between(x, y, max(y), color=light_green_color)

ax.text(0.65, 0.9, text['bmap'], size=25, color=blue_color)
ax.text(0.82, 0.80, text['bmap_sub'], size=8, color=blue_color)

prompts_size=15
ax.text(0.5, 0.5, text['prompts'], size=prompts_size, color=blue_color)
ax.text(0.5, 0.45, text['prompts_succeed'], size=prompts_size*0.5, color=green_color)
ax.text(0.08, 0.2, text['prompts'], size=prompts_size, color=blue_color)
ax.text(0.08, 0.15, text['prompts_fail'], size=prompts_size*0.5, color=red_color)

# set x axis ticks names
xticks = [''] * len(x)
xticks[10], xticks[-10] = text['ability_axis']
plt.xticks(x, xticks)
# set y axis ticks names
yticks = [''] * len(y)
yticks[6], yticks[-1] = text['motivation_axis']
plt.yticks(y, yticks)

# set axis' limits
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.0])

tp = np.array((0.28, 0.45))
trans_angle = plt.gca().transData.transform_angles(np.array((315,)),
                                                   tp.reshape((1, 2)))[0]
plt.text(tp[0], tp[1], text['action_line'], fontsize=12,
         rotation=trans_angle, rotation_mode='anchor')
plt.show()
Figure 1: Image based in the Fogg Behavior Model (as presented in the book).

Looking at the graphic we can see that if there is a prompt for a behavior, but it is something too hard to do or if the motivation is low, the behavior will fail.

On the contrary, if the behavior is easy to do and we have the motivation when the prompt surges the behavior will be a success!

Thinking in the process of building new habits, we could use this model as a guide on how to develop a new one. After choosing the right behavior, make sure you have the right motivation for it and make it really easy to do it! In other words, you should make it as simple and small so that there is no chance you will not do it. In this way, each time you are successful with your behavior you are taking a step in making it a new habit.

A similar conclusion can be thought in the process of stopping a bad habit, make it really hard to do, so that even when the prompt arises you will be less likely to do it.

This is the suggested template by B. J. Fogg to create a new behavior:

After I ______, I will ______. Then, I celebrate!

The celebration is an important step for you to connect the behavior with something positive that should be repeated.

If you think the behavior you tried to create is not working, try this troubleshoot method:

  1. Check prompt
  2. Check ability
  3. Check motivation

And keep trying out and learning more about yourself, one small step at a time.

References

Back to top ]]> python matplotlib productivity https://hygorxaraujo.com/posts/2020-03-08-explaining-the-fogg-behavior-model.html Sun, 08 Mar 2020 03:00:00 GMT How to check missing values in pandas Hygor X. Araújo https://hygorxaraujo.com/posts/2018-06-30-pandas-check-nan.html In this article I would like to describe how to find NaN values in a pandas DataFrame. This kind of operation can be very useful given that is common to find datasets with missing or incorrect data values.

I will be using the numpy package to generate some data with NaN values.

Import necessary packages

import pandas as pd
import numpy as np
import platform
print(f'Python version: {platform.python_version()} ({platform.python_implementation()})')
print(f'Pandas version: {pd.__version__}')
print(f'Numpy version: {np.__version__}')
Python version: 3.12.12 (CPython)
Pandas version: 3.0.0
Numpy version: 2.4.2

Generate data with NaN values

num_nan = 25 # number of NaN values wanted in the generated data
np.random.seed(6765431)  # set a seed for reproducibility
A = np.random.randn(10, 10)
print(A)
[[-1.56132314 -0.16954058 -0.17845422 -1.33689111 -0.19185078 -1.18617765
   0.44499302 -0.61209568  0.31170935  1.4127548 ]
 [ 0.85330488  0.68517546 -1.10140989  0.84918019  0.72802961 -0.35161197
   0.73519152  1.13145412  0.53231247  0.78103143]
 [-0.81614324  0.15906898  0.49940119 -0.09319255 -1.07837721 -0.76053341
   0.73622083 -0.45518154 -0.69194032  1.02550409]
 [-1.96339975  0.07593331 -0.16798377 -1.20398958  0.88333656  1.17908422
   0.26324698 -2.65442248 -0.31583796 -0.16065732]
 [-1.24321376 -0.89816898  0.02824671  0.15304093  0.56505667 -0.78115883
   0.74504467  1.14025258 -0.04518221 -0.83908358]
 [ 1.00967019  0.84240102  1.15043436 -0.40120489  0.00664105 -1.23247563
   0.64738343  1.66096762 -0.92556683  0.47575796]
 [ 0.96516278  1.11158059 -0.82155143  0.88900313  2.16943761 -2.05250161
   2.40156233  0.92453867 -0.24437783 -2.91029265]
 [-0.86492662  0.82443151 -0.48246862 -1.05183143 -1.15272524 -0.77170733
   0.07177233  1.02820181 -2.08947076  0.89859677]
 [-0.07263982 -0.56840867  1.30910275 -0.52846822  0.06019191 -0.61000727
   0.40782356 -0.36124333 -1.54522486 -0.07891861]
 [-1.96361682 -1.06315325 -0.45582138 -0.74566868  1.27579529 -2.46306005
   0.57022673 -0.02793746  0.78652775  1.27690195]]
# Set random values to nan
A.ravel()[np.random.choice(A.size, num_nan, replace=False)] = np.nan
print(A)
[[-1.56132314 -0.16954058 -0.17845422 -1.33689111 -0.19185078 -1.18617765
          nan -0.61209568  0.31170935  1.4127548 ]
 [ 0.85330488  0.68517546         nan  0.84918019         nan -0.35161197
   0.73519152         nan  0.53231247  0.78103143]
 [-0.81614324  0.15906898  0.49940119         nan -1.07837721 -0.76053341
   0.73622083         nan -0.69194032  1.02550409]
 [-1.96339975  0.07593331         nan -1.20398958  0.88333656         nan
   0.26324698         nan -0.31583796 -0.16065732]
 [-1.24321376 -0.89816898  0.02824671  0.15304093  0.56505667 -0.78115883
   0.74504467  1.14025258 -0.04518221 -0.83908358]
 [ 1.00967019  0.84240102         nan -0.40120489  0.00664105         nan
   0.64738343  1.66096762 -0.92556683  0.47575796]
 [ 0.96516278         nan -0.82155143  0.88900313  2.16943761         nan
   2.40156233         nan -0.24437783         nan]
 [-0.86492662  0.82443151 -0.48246862 -1.05183143 -1.15272524 -0.77170733
   0.07177233  1.02820181 -2.08947076         nan]
 [-0.07263982         nan  1.30910275 -0.52846822  0.06019191 -0.61000727
   0.40782356 -0.36124333         nan         nan]
 [        nan         nan         nan         nan  1.27579529 -2.46306005
          nan         nan  0.78652775  1.27690195]]
# Create a DataFrame from the generated data
df = pd.DataFrame(A)
df
0 1 2 3 4 5 6 7 8 9
0 -1.561323 -0.169541 -0.178454 -1.336891 -0.191851 -1.186178 NaN -0.612096 0.311709 1.412755
1 0.853305 0.685175 NaN 0.849180 NaN -0.351612 0.735192 NaN 0.532312 0.781031
2 -0.816143 0.159069 0.499401 NaN -1.078377 -0.760533 0.736221 NaN -0.691940 1.025504
3 -1.963400 0.075933 NaN -1.203990 0.883337 NaN 0.263247 NaN -0.315838 -0.160657
4 -1.243214 -0.898169 0.028247 0.153041 0.565057 -0.781159 0.745045 1.140253 -0.045182 -0.839084
5 1.009670 0.842401 NaN -0.401205 0.006641 NaN 0.647383 1.660968 -0.925567 0.475758
6 0.965163 NaN -0.821551 0.889003 2.169438 NaN 2.401562 NaN -0.244378 NaN
7 -0.864927 0.824432 -0.482469 -1.051831 -1.152725 -0.771707 0.071772 1.028202 -2.089471 NaN
8 -0.072640 NaN 1.309103 -0.528468 0.060192 -0.610007 0.407824 -0.361243 NaN NaN
9 NaN NaN NaN NaN 1.275795 -2.463060 NaN NaN 0.786528 1.276902

Check for NaN values

Now that we have some data to operate on let’s see the different ways we can check for missing values.

There are two methods of the DataFrame object that can be used: DataFrame#isna() and DataFrame#isnull(). But if you check the source code it seems that isnull() is only an alias for the isna() method. To keep it simple I will only use the isna() method as we would get the same result using isnull().

df.isna()
0 1 2 3 4 5 6 7 8 9
0 False False False False False False True False False False
1 False False True False True False False True False False
2 False False False True False False False True False False
3 False False True False False True False True False False
4 False False False False False False False False False False
5 False False True False False True False False False False
6 False True False False False True False True False True
7 False False False False False False False False False True
8 False True False False False False False False True True
9 True True True True False False True True False False

As it can be seen above when we use the isna() method it returns a DataFrame with boolean values, where True indicates NaN values and False otherwise.

If we wanted to know how many missing values there are on each row or column we could use the DataFrame#sum() method:

df.isna().sum(axis='rows')  # 'rows' or 0
0    1
1    3
2    4
3    2
4    1
5    3
6    2
7    5
8    1
9    3
dtype: int64
df.isna().sum(axis='columns')  # 'columns' or 1
0    1
1    3
2    2
3    3
4    0
5    2
6    4
7    1
8    3
9    6
dtype: int64

To simply know the total number of missing values we can call sum() again:

df.isna().sum().sum()
np.int64(25)

If we simply wanna know if there is any missing value with no care for the quantity we can simply use the any() method:

df.isna().any()  # can also receive axis='rows' or 'columns'
0    True
1    True
2    True
3    True
4    True
5    True
6    True
7    True
8    True
9    True
dtype: bool

Calling it again we have a single boolean output:

df.isna().any().any()
np.True_

Besides the isna() method we also have the notna() method which is its boolean inverse. Applying it we can get the number of values that are not missing or simply if all values are not missing (but using the all() method instead of any()).

print(df.notna().sum().sum())  # not missing
print(df.notna().all().all())
75
False

Note 1: in the examples, it was used the DataFrame methods to check for missing values, but the pandas package has its own functions with the same purpose that can be applied to other objects. Example:

print(pd.isna([1, 2, np.nan]))
print(pd.notna([1, 2, np.nan]))
[False False  True]
[ True  True False]

Note 2: the methods applied here on DataFrame objects are also available for Series and Index objects.

Time comparison

Comparing the time taken by the two methods we can see that using any() is faster but sum() will give us the additional information about how many missing values there are.

%timeit df.isna().any().any()
158 μs ± 19.5 μs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
%timeit df.isna().sum().sum()
107 μs ± 4.85 μs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

Dealing with missing values

Two easy ways to deal with missing values are removing them or filling them with some value. These can be achieved with the dropna() and fillna() methods.

The dropna() method will return a DataFrame without the rows and columns containing missing values.

df.dropna()
0 1 2 3 4 5 6 7 8 9
4 -1.243214 -0.898169 0.028247 0.153041 0.565057 -0.781159 0.745045 1.140253 -0.045182 -0.839084

The fillna() method will return a DataFrame with the missing values filled with a specified value.

df.fillna(value=5)
0 1 2 3 4 5 6 7 8 9
0 -1.561323 -0.169541 -0.178454 -1.336891 -0.191851 -1.186178 5.000000 -0.612096 0.311709 1.412755
1 0.853305 0.685175 5.000000 0.849180 5.000000 -0.351612 0.735192 5.000000 0.532312 0.781031
2 -0.816143 0.159069 0.499401 5.000000 -1.078377 -0.760533 0.736221 5.000000 -0.691940 1.025504
3 -1.963400 0.075933 5.000000 -1.203990 0.883337 5.000000 0.263247 5.000000 -0.315838 -0.160657
4 -1.243214 -0.898169 0.028247 0.153041 0.565057 -0.781159 0.745045 1.140253 -0.045182 -0.839084
5 1.009670 0.842401 5.000000 -0.401205 0.006641 5.000000 0.647383 1.660968 -0.925567 0.475758
6 0.965163 5.000000 -0.821551 0.889003 2.169438 5.000000 2.401562 5.000000 -0.244378 5.000000
7 -0.864927 0.824432 -0.482469 -1.051831 -1.152725 -0.771707 0.071772 1.028202 -2.089471 5.000000
8 -0.072640 5.000000 1.309103 -0.528468 0.060192 -0.610007 0.407824 -0.361243 5.000000 5.000000
9 5.000000 5.000000 5.000000 5.000000 1.275795 -2.463060 5.000000 5.000000 0.786528 1.276902

References:

Back to top ]]> python pandas numpy https://hygorxaraujo.com/posts/2018-06-30-pandas-check-nan.html Sat, 30 Jun 2018 03:00:00 GMT