Kae Nagano - Fab Futures - Data Science

Home About

Class

- Machine Learning

Assignment　　

• Fit a machine learning model to your data

Approach

1. Understand XOR examples

XOR example @ class

import jax
import jax.numpy as jnp
from jax import random, grad, jit
#
# init random key
#
key = random.PRNGKey(0)
#
# XOR training data
#
X = jnp.array([[0.,0.],[0.,1.],[1.,0.],[1.,1.]],dtype=jnp.float32)
y = jnp.array([[0.],[1.],[1.],[0.]],dtype=jnp.float32)

#
# parameter initialization
#
def init_params(key,hidden=2):
    k1,k2 = random.split(key)
    W1 = 0.1*random.normal(k1,(2,hidden))
    b1 = jnp.zeros((hidden,))
    W2 = 0.1*random.normal(k2,(hidden,1))
    b2 = jnp.zeros((1,))
    return (W1,b1,W2,b2)
#
# forward pass
#
def forward(params,x):
    W1,b1,W2,b2 = params
    h = jnp.tanh(x@W1+b1)
    o = jax.nn.sigmoid(h@W2+b2)
    return o
#
# loss function
#
def loss(params):
    y_pred = forward(params,X)
    return jnp.mean((y_pred-y)**2)
#
# update weights
#
@jit
def update(params,rate=0.5):
    g = grad(loss)(params)
    return jax.tree.map(lambda p,gp:p-rate*gp,params,g)
#
# initialize parameters
#
params = init_params(key)
#
# training steps
#
for step in range(5000):
    params = update(params,rate=0.5)
    if step % 1000 == 0:
        print(f"step {step:4d} loss={loss(params):.6f}")
#
# evaluate fit
#
pred = forward(params,X)
print("\nPredictions:")
print(jnp.concatenate([X,pred],axis=1))

step    0 loss=0.250014
step 1000 loss=0.011100
step 2000 loss=0.001649
step 3000 loss=0.000851
step 4000 loss=0.000568

Predictions:
[[0.         0.         0.02282555]
 [0.         1.         0.98264414]
 [1.         0.         0.98267007]
 [1.         1.         0.02402291]]

---

Memo:　　

• [PRNG](pseudo random number generation): seed => key => random number
• JIT: Just-in-time compilation
• jax.jit

2. MLP @ Fab Academy Data

Based on the XOR code, I tried MLP using Fab Academy Data.

• MLP　　　
• Standardize：　Standardizes input to zero mean and unit variance.
• A CSV file loaded with Pandas needs to be converted into a NumPy array.
• I reshaped X and Y to (14, 1) so that they match the expected input format for the model: a 2-dimensional column vector.
• Normalization / Standardization -　manual standardization: X = (X - X.mean()) / X.std()
  • standardize function

Example of a failure case　：　Using the sigmoid activation function　　　　

Because I reused the XOR code example, the sigmoid activation function was still selected. As a result, the MLP output ended up being zero.

# import jax
import jax.numpy as jnp
from jax import random, grad, jit
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from jax.nn import standardize

df = pd.read_csv("datasets/FA_students_graduates_2012_2025.csv")
X = df["year"].to_numpy()
Y = df["students"].to_numpy()

X = X.reshape(14,1)
Y = Y.reshape(14,1)

X = (X - X.mean()) / X.std()  
#X = standardize(X, axis=0)


X = jnp.array(X)
Y = jnp.array(Y)

key = random.PRNGKey(0)

def init_params(key,hidden=6):   
    k1,k2 = random.split(key)
    W1 = 0.1*random.normal(k1,(1,hidden))
    b1 = jnp.zeros((hidden,))
    W2 = 0.1*random.normal(k2,(hidden,1))
    b2 = jnp.zeros((1,))
    return (W1,b1,W2,b2)

@jit
def forward(params,x):
    W1,b1,W2,b2 = params
    h = jnp.tanh(x@W1+b1)
    o = jax.nn.sigmoid(h@W2+b2)  
    return o

@jit
def loss(params):
    y_pred = forward(params,X)
    return jnp.mean((y_pred-Y)**2)
    
@jit
def update(params,rate=0.5):
    g = grad(loss)(params)
    return jax.tree.map(lambda p,gp:p-rate*gp,params,g)

params = init_params(key)

for step in range(5000):
#    params = update(params,rate=0.5)
    params = update(params,rate=0.01)
    if step % 500 == 0:
        print(f"step {step:4d} loss={loss(params):.6f}")
        
pred = forward(params,X)
plt.figure(figsize=(10,5))
plt.scatter(X,Y, label="data", color="blue")

plt.plot(X, pred, "r-", label="MLP fit", linewidth=2)
plt.xlabel("Year")
plt.ylabel("Students")
plt.title("FabAcademy Students — MLPfit Hidden layer = 6 ")
plt.grid(True)
plt.legend()
plt.show()

step    0 loss=43663.925781
step  500 loss=43565.906250
step 1000 loss=43565.882812
step 1500 loss=43565.863281
step 2000 loss=43565.863281
step 2500 loss=43565.863281
step 3000 loss=43565.863281
step 3500 loss=43565.863281
step 4000 loss=43565.863281
step 4500 loss=43565.863281

A successful example without an activation function　　　

Because this is a regression task, the output layer should not use an activation function. The model needs to produce continuous values, and applying an activation would unnecessarily restrict the output range.　 Changing the number of hidden layers affects the output.　In the example where I increased it to 10, the model became overfitted.　　

Case: Hidden layer = 6

# import jax
import jax.numpy as jnp
from jax import random, grad, jit
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from jax.nn import standardize

df = pd.read_csv("datasets/FA_students_graduates_2012_2025.csv")
X = df["year"].to_numpy()
Y = df["students"].to_numpy()

X = X.reshape(14,1)
Y = Y.reshape(14,1)

X = (X - X.mean()) / X.std()  # こっちの例もある
#X = standardize(X, axis=0)


X = jnp.array(X)
Y = jnp.array(Y)

key = random.PRNGKey(0)

def init_params(key,hidden=6):   #Layer数でけっこうかわるな
    k1,k2 = random.split(key)
    W1 = 0.1*random.normal(k1,(1,hidden))
    b1 = jnp.zeros((hidden,))
    W2 = 0.1*random.normal(k2,(hidden,1))
    b2 = jnp.zeros((1,))
    return (W1,b1,W2,b2)

@jit
def forward(params,x):
    W1,b1,W2,b2 = params
    h = jnp.tanh(x@W1+b1)
    o = h@W2+b2  # 
    #o = jax.nn.sigmoid(h@W2+b2)  
    return o

@jit
def loss(params):
    y_pred = forward(params,X)
    return jnp.mean((y_pred-Y)**2)
    
@jit
def update(params,rate=0.5):
    g = grad(loss)(params)
    return jax.tree.map(lambda p,gp:p-rate*gp,params,g)

params = init_params(key)

for step in range(5000):
#    params = update(params,rate=0.5)
    params = update(params,rate=0.01)
    if step % 500 == 0:
        print(f"step {step:4d} loss={loss(params):.6f}")
        
pred = forward(params,X)
plt.figure(figsize=(10,5))
plt.scatter(X,Y, label="data", color="blue")

plt.plot(X, pred, "r-", label="MLP fit", linewidth=2)
plt.xlabel("Year")
plt.ylabel("Students")
plt.title("FabAcademy Students — MLPfit Hidden layer = 6 ")
plt.grid(True)
plt.legend()
plt.show()

step    0 loss=42225.527344
step  500 loss=782.627869
step 1000 loss=757.749756
step 1500 loss=756.523315
step 2000 loss=755.269470
step 2500 loss=758.228149
step 3000 loss=763.090576
step 3500 loss=754.070435
step 4000 loss=752.267395
step 4500 loss=751.753479

Case: Hidden layer = 10

# import jax
import jax.numpy as jnp
from jax import random, grad, jit
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from jax.nn import standardize

df = pd.read_csv("datasets/FA_students_graduates_2012_2025.csv")
X = df["year"].to_numpy()
Y = df["students"].to_numpy()

X = X.reshape(14,1)
Y = Y.reshape(14,1)

X = (X - X.mean()) / X.std()  # こっちの例もある
#X = standardize(X, axis=0)


X = jnp.array(X)
Y = jnp.array(Y)

key = random.PRNGKey(0)

def init_params(key,hidden=10):   #Layer数でけっこうかわるな
    k1,k2 = random.split(key)
    W1 = 0.1*random.normal(k1,(1,hidden))
    b1 = jnp.zeros((hidden,))
    W2 = 0.1*random.normal(k2,(hidden,1))
    b2 = jnp.zeros((1,))
    return (W1,b1,W2,b2)

@jit
def forward(params,x):
    W1,b1,W2,b2 = params
    h = jnp.tanh(x@W1+b1)
    o = h@W2+b2  # これこれ
    #o = jax.nn.sigmoid(h@W2+b2)  
    return o

@jit
def loss(params):
    y_pred = forward(params,X)
    return jnp.mean((y_pred-Y)**2)
    
@jit
def update(params,rate=0.5):
    g = grad(loss)(params)
    return jax.tree.map(lambda p,gp:p-rate*gp,params,g)

params = init_params(key)

for step in range(5000):
#    params = update(params,rate=0.5)
    params = update(params,rate=0.01)
    if step % 500 == 0:
        print(f"step {step:4d} loss={loss(params):.6f}")
        
pred = forward(params,X)
plt.figure(figsize=(10,5))
plt.scatter(X,Y, label="data", color="blue")

plt.plot(X, pred, "r-", label="MLP fit", linewidth=2)
plt.xlabel("Year")
plt.ylabel("Students")
plt.title("FabAcademy Students — MLPfit ")
plt.grid(True)
plt.legend()
plt.show()

step    0 loss=42178.984375
step  500 loss=314.968414
step 1000 loss=309.107910
step 1500 loss=310.286682
step 2000 loss=284.416748
step 2500 loss=273.091248
step 3000 loss=272.860931
step 3500 loss=272.565643
step 4000 loss=272.293732
step 4500 loss=176.202499

3. CNN @ Fab Academy Data　　

I decided to experiment with a CNN because I enjoy working with image-based methods. Initially, I considered using the Fab Academy final project images for tagging and classification. However, these images are not standardized—each page has a different layout, and text and images are mixed together. To use them in a CNN, I would need to crop out important regions and perform extensive preprocessing, which would be time-consuming.

Instead, I chose to convert the GitLab commit time-series data—previously used for visualization—into image format and apply a CNN to extract features. This idea was inspired by research that converts music spectrograms into images for neural-network training.

After extracting features from the commit patterns with a CNN, my plan is to perform clustering to categorize students into different learning or working styles.

Step1: Create Image　　

• fill_between　　
• Savefig
• Close figure window
• Delete label/axis

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

df = pd.read_csv("datasets/fabacademy_commit_weekly_summary.csv")

# Week name
week_cols = [col for col in df.columns if col.startswith("week_")]
weeks = list(range(1, len(week_cols) + 1))  
print(weeks)
print(commits, commits.dtype)

output_dir = "datasets/commitsImage"

for idx, row in df.iterrows():
    if idx >= 3:
        break
    commits = row[week_cols].astype(float).values   #cast
    #commits = row[week_cols].values   
    commits = commits/commits.max()
    #student = row["student_name"]
    #lab = row["lab_name"]
    fig, ax = plt.subplots(figsize=(3, 3), dpi=100)
    ax.plot(weeks, commits)
    ax.fill_between(weeks, commits)
    ax.axis("off")
    plt.show()
    #plt.savefig(f"{output_dir}/img_{idx}.jpg")

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]
[0.         1.         0.43478261 0.60869565 0.26086957 0.
 0.         0.         0.         0.08695652 0.         0.
 0.         0.         0.2173913  0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.        ] float64

Step2: Adjust Image Size

• Mnist image : 28x28 pixel
• Reference
  • figsize=(w_in, h_in) :inch
  • dpi: dot / inch
  • w_px = w_in *dpi , h_px = h_in * dpi
• To make 28 x 28 pixel: w_in, h_in = 0.28
• Color to Glay scale

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

df = pd.read_csv("datasets/fabacademy_commit_weekly_summary.csv")

# Week name
week_cols = [col for col in df.columns if col.startswith("week_")]
weeks = list(range(1, len(week_cols) + 1))  
#print(weeks)
#print(commits, commits.dtype)

output_dir = "datasets/commitsImage"

for idx, row in df.iterrows():
    if idx >= 3:
        break
    commits = row[week_cols].astype(float).values   #cast
    #commits = row[week_cols].values   
    commits = commits/commits.max()
    #student = row["student_name"]
    #lab = row["lab_name"]
    fig, ax = plt.subplots(figsize=(0.28, 0.28), dpi=100)
    ax.plot(weeks, commits,color="black")
    ax.fill_between(weeks, commits,color="black", alpha=1.0)
    ax.set_facecolor("white")
    ax.axis("off")
    plt.show()
    #plt.savefig(f"{output_dir}/img_{idx}.jpg")

Step3: Save files

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

df = pd.read_csv("datasets/fabacademy_commit_weekly_summary.csv")

# Week name
week_cols = [col for col in df.columns if col.startswith("week_")]
weeks = list(range(1, len(week_cols) + 1))  
print(weeks)
print(commits, commits.dtype)

output_dir = "datasets/commitsImage"

for idx, row in df.iterrows():
    #if idx >= 3:
    #    break
    commits = row[week_cols].astype(float).values   #cast
    #commits = row[week_cols].values   
    max=commits.max()
    if(max>0):
        commits = commits/max
    #student = row["student_name"]
    #lab = row["lab_name"]
    fig, ax = plt.subplots(figsize=(0.28, 0.28), dpi=100)
    ax.plot(weeks, commits,color="black")
    ax.fill_between(weeks, commits,color="black", alpha=1.0)
    ax.set_facecolor("white")
    ax.axis("off")
    #plt.show()
    plt.savefig(f"{output_dir}/img_{idx}.jpg", dpi=100)
    plt.close(fig)

print("Images are saved")

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]
[0.         1.         0.43478261 0.60869565 0.26086957 0.
 0.         0.         0.         0.08695652 0.         0.
 0.         0.         0.2173913  0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.        ] float64
Images are saved

Step4 Pytorch Dataset

I referred to the official PyTorch Data set and CNN examples.

https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset　　　　　 All datasets that represent an iterable of data samples should subclass it.

Reference

• Pytorch CNN
• Pytorch Book
• Dataset Class

Dataset

!pip install torch torchvision

Collecting torch
  Using cached torch-2.9.1-cp313-cp313-manylinux_2_28_aarch64.whl.metadata (30 kB)
Collecting torchvision
  Using cached torchvision-0.24.1-cp313-cp313-manylinux_2_28_aarch64.whl.metadata (5.9 kB)
Collecting filelock (from torch)
  Using cached filelock-3.20.0-py3-none-any.whl.metadata (2.1 kB)
Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/lib/python3.13/site-packages (from torch) (4.15.0)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.13/site-packages (from torch) (80.9.0)
Requirement already satisfied: sympy>=1.13.3 in /opt/conda/lib/python3.13/site-packages (from torch) (1.14.0)
Requirement already satisfied: networkx>=2.5.1 in /opt/conda/lib/python3.13/site-packages (from torch) (3.5)
Requirement already satisfied: jinja2 in /opt/conda/lib/python3.13/site-packages (from torch) (3.1.6)
Requirement already satisfied: fsspec>=0.8.5 in /opt/conda/lib/python3.13/site-packages (from torch) (2025.9.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.13/site-packages (from torchvision) (2.3.3)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/lib/python3.13/site-packages (from torchvision) (11.3.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/lib/python3.13/site-packages (from sympy>=1.13.3->torch) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.13/site-packages (from jinja2->torch) (3.0.3)
Using cached torch-2.9.1-cp313-cp313-manylinux_2_28_aarch64.whl (104.1 MB)
Using cached torchvision-0.24.1-cp313-cp313-manylinux_2_28_aarch64.whl (2.3 MB)
Using cached filelock-3.20.0-py3-none-any.whl (16 kB)
Installing collected packages: filelock, torch, torchvision
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3 [torchvision] [torchvision]
Successfully installed filelock-3.20.0 torch-2.9.1 torchvision-0.24.1

from torch.utils.data import Dataset
from torchvision import transforms
from PIL import Image
from pathlib import Path

class FabAcademyCommitDataset(Dataset):
    def __init__(self, root_dir, transform=None):
        self.root_dir = root_dir
        self.files = list(Path(root_dir).glob("*.jpg"))
        self.transform = transform

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        img = Image.open(self.files[idx])
        if self.transform:
            img = self.transform(img)
        return img, 0

from torchvision import transforms

transform = transforms.Compose([
    transforms.Grayscale(),  
    transforms.ToTensor(),    
])

Show the three original images

FA_commit_dataset = FabAcademyCommitDataset(
    root_dir="datasets/commitsImage",
    transform=transform
)

import matplotlib.pyplot as plt


fig = plt.figure(figsize=(10, 4))

for i, (img, label) in enumerate(FA_commit_dataset):
    if i >= 3:
        break

    ax = plt.subplot(1, 3, i + 1)
    plt.tight_layout()
    ax.set_title(f"Sample #{i}")
    ax.axis("off")

    # ToTensor() してあるので (1, H, W) → (H, W) へ
    ax.imshow(img.squeeze(0), cmap="gray")

plt.show()

Reference: https://docs.pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html

optimizer : SGD

loss: CrossEntropyLoss　　

Step5 Define and train the model

Since the CNN code was originally designed for the CIFAR-10 dataset, I scaled my images to 32×32 as well (at least for this initial experiment).　　　 The model is mostly the same as the original, but since I converted the images to grayscale, I changed the input from 3 channels to 1 channel.

import torch
import torchvision.transforms as transforms
from torch.utils.data import Dataset, DataLoader
from PIL import Image
from pathlib import Path

transform = transforms.Compose([
    transforms.Grayscale(),     # 上から実行
    transforms.Resize((32,32)), # MNISTにあわせたけど、もともと３２でつくりなおそうかな
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])


class FabAcademyCommitDataset(Dataset):
    def __init__(self, root_dir, transform=None):
        self.root_dir = root_dir
        self.files = list(Path(root_dir).glob("*.jpg"))
        self.transform = transform

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        img = Image.open(self.files[idx])
        if self.transform:
            img = self.transform(img)
        return img, 0

trainset = FabAcademyCommitDataset(
    root_dir="datasets/commitsImage",
    transform=transform
)

batch_size = 4

trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=2) #オリジナルのでよさそう
#trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0)

inputs, labels = next(iter(trainloader))
print("BATCH SHAPE:", inputs.shape)  #1508/4  bactch=4

testset = trainset   # あとでテストをつくる > 来週のために少しとっておくか
testloader = trainloader

# ---　Define a Convolutional Neural Network ---

import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)   # ここだけ　１chにしてる
        self.pool  = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1   = nn.Linear(16 * 5 * 5, 120)
        self.fc2   = nn.Linear(120, 84)
        self.fc3   = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()

print(net)

# --- Define a Loss function and optimizer ---

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)


# --- Train the network ---
print("LEN trainloader:", len(trainloader))

#for epoch in range(2): # loop over the dataset multiple times
for epoch in range(1): # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        #print("LOOP = ", i)
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data
        
        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 10 == 0:
            print(f"[{epoch + 1}, {i + 1:5d}] loss: {running_loss / (i+1):.3f}")
            running_loss = 0.0

print("Finished Training")

BATCH SHAPE: torch.Size([4, 1, 32, 32])
Net(
  (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))
  (fc1): Linear(in_features=400, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=84, bias=True)
  (fc3): Linear(in_features=84, out_features=10, bias=True)
)
LEN trainloader: 378
[1,     1] loss: 2.242
[1,    11] loss: 2.015
[1,    21] loss: 1.013
[1,    31] loss: 0.647
[1,    41] loss: 0.455
[1,    51] loss: 0.333
[1,    61] loss: 0.239
[1,    71] loss: 0.144
[1,    81] loss: 0.041
[1,    91] loss: 0.002
[1,   101] loss: 0.000
[1,   111] loss: 0.000
[1,   121] loss: 0.000
[1,   131] loss: 0.000
[1,   141] loss: 0.000
[1,   151] loss: 0.000
[1,   161] loss: 0.000
[1,   171] loss: 0.000
[1,   181] loss: 0.000
[1,   191] loss: 0.000
[1,   201] loss: 0.000
[1,   211] loss: 0.000
[1,   221] loss: 0.000
[1,   231] loss: 0.000
[1,   241] loss: 0.000
[1,   251] loss: 0.000
[1,   261] loss: 0.000
[1,   271] loss: 0.000
[1,   281] loss: 0.000
[1,   291] loss: 0.000
[1,   301] loss: 0.000
[1,   311] loss: 0.000
[1,   321] loss: 0.000
[1,   331] loss: 0.000
[1,   341] loss: 0.000
[1,   351] loss: 0.000
[1,   361] loss: 0.000
[1,   371] loss: 0.000
Finished Training

Step6 Feature Visualization

• Intermediate-layer visualization
• hook

import torch
import torchvision.transforms as transforms
from torch.utils.data import Dataset, DataLoader
from PIL import Image
from pathlib import Path

transform = transforms.Compose([
    transforms.Grayscale(),     # 上から実行
    transforms.Resize((32,32)), # MNISTにあわせたけど、もともと３２でつくりなおそうかな
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])


class FabAcademyCommitDataset(Dataset):
    def __init__(self, root_dir, transform=None):
        self.root_dir = root_dir
        self.files = list(Path(root_dir).glob("*.jpg"))
        self.transform = transform

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        img = Image.open(self.files[idx])
        if self.transform:
            img = self.transform(img)
        return img, 0

trainset = FabAcademyCommitDataset(
    root_dir="datasets/commitsImage",
    transform=transform
)

batch_size = 4

trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=2) #オリジナルのでよさそう
#trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0)

inputs, labels = next(iter(trainloader))
print("BATCH SHAPE:", inputs.shape)  #1508/4  bactch=4

testset = trainset   # あとでテストをつくる > 来週のために少しとっておくか
testloader = trainloader

# ---　Define a Convolutional Neural Network ---

import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)   # ここだけ　１chにしてる
        self.pool  = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1   = nn.Linear(16 * 5 * 5, 120)
        self.fc2   = nn.Linear(120, 84)
        self.fc3   = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()

print(net)

# --- feature map hook  ---
feature_maps = {}

def register_activation_hook(name):
    def hook(module, input, output):
        feature_maps[name] = output.detach()
    return hook

net.conv1.register_forward_hook(register_activation_hook("conv1"))
net.conv2.register_forward_hook(register_activation_hook("conv2"))


# --- Define a Loss function and optimizer ---

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)


# --- Train the network ---
print("LEN trainloader:", len(trainloader))

#for epoch in range(2): # loop over the dataset multiple times
for epoch in range(1): # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        #print("LOOP = ", i)
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data
        
        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 10 == 0:
            print(f"[{epoch + 1}, {i + 1:5d}] loss: {running_loss / (i+1):.3f}")
            running_loss = 0.0

print("Finished Training")

# --- Visualize feature map ---
import matplotlib.pyplot as plt

def visualize_feature_map(name):
    fmap = feature_maps[name]           
    fmap = fmap.squeeze(0).cpu().numpy()   # [C, H, W]

    num_channels = fmap.shape[0]
    cols = 8
    rows = (num_channels + cols - 1) // cols

    fig = plt.figure(figsize=(16, 2*rows))
    for i in range(num_channels):
        ax = fig.add_subplot(rows, cols, i+1)
        ax.imshow(fmap[i], cmap='gray')
        ax.axis("off")

    plt.suptitle(f"{name} feature maps")
    plt.show()
    plt.close(fig)


visualize_feature_map("conv1")
visualize_feature_map("conv2")

BATCH SHAPE: torch.Size([4, 1, 32, 32])
Net(
  (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))
  (fc1): Linear(in_features=400, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=84, bias=True)
  (fc3): Linear(in_features=84, out_features=10, bias=True)
)
LEN trainloader: 378
[1,     1] loss: 2.347
[1,    11] loss: 2.105
[1,    21] loss: 1.055
[1,    31] loss: 0.673
[1,    41] loss: 0.473
[1,    51] loss: 0.348
[1,    61] loss: 0.253
[1,    71] loss: 0.165
[1,    81] loss: 0.058
[1,    91] loss: 0.005
[1,   101] loss: 0.000
[1,   111] loss: 0.000
[1,   121] loss: 0.000
[1,   131] loss: 0.000
[1,   141] loss: 0.000
[1,   151] loss: 0.000
[1,   161] loss: 0.000
[1,   171] loss: 0.000
[1,   181] loss: 0.000
[1,   191] loss: 0.000
[1,   201] loss: 0.000
[1,   211] loss: 0.000
[1,   221] loss: 0.000
[1,   231] loss: 0.000
[1,   241] loss: 0.000
[1,   251] loss: 0.000
[1,   261] loss: 0.000
[1,   271] loss: 0.000
[1,   281] loss: 0.000
[1,   291] loss: 0.000
[1,   301] loss: 0.000
[1,   311] loss: 0.000
[1,   321] loss: 0.000
[1,   331] loss: 0.000
[1,   341] loss: 0.000
[1,   351] loss: 0.000
[1,   361] loss: 0.000
[1,   371] loss: 0.000
Finished Training

---

Because the images are very simple, the loss quickly dropped to zero, indicating possible overfitting.

• I reduced the number of channels (Conv1: 6 → 3, Conv2: 16 → 6).
• Although the loss still becomes zero, reducing the channels made the feature maps visually clearer.

import torch
import torchvision.transforms as transforms
from torch.utils.data import Dataset, DataLoader
from PIL import Image
from pathlib import Path

transform = transforms.Compose([
    transforms.Grayscale(),     # 上から実行
    transforms.Resize((32,32)), # MNISTにあわせたけど、もともと３２でつくりなおそうかな
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])


class FabAcademyCommitDataset(Dataset):
    def __init__(self, root_dir, transform=None):
        self.root_dir = root_dir
        self.files = list(Path(root_dir).glob("*.jpg"))
        self.transform = transform

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        img = Image.open(self.files[idx])
        if self.transform:
            img = self.transform(img)
        return img, 0

trainset = FabAcademyCommitDataset(
    root_dir="datasets/commitsImage",
    transform=transform
)

batch_size = 4

trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=2) #オリジナルのでよさそう
#trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0)

inputs, labels = next(iter(trainloader))
print("BATCH SHAPE:", inputs.shape)  #1508/4  bactch=4

testset = trainset   # あとでテストをつくる > 来週のために少しとっておくか
testloader = trainloader

# ---　Define a Convolutional Neural Network ---

import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 3, 5)   # 
        self.pool  = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(3, 6, 5)
        self.fc1   = nn.Linear(6 * 5 * 5, 120)
        self.fc2   = nn.Linear(120, 84)
        self.fc3   = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()

print(net)

# --- feature map hook  ---
feature_maps = {}

def register_activation_hook(name):
    def hook(module, input, output):
        feature_maps[name] = output.detach()
    return hook

net.conv1.register_forward_hook(register_activation_hook("conv1"))
net.conv2.register_forward_hook(register_activation_hook("conv2"))


# --- Define a Loss function and optimizer ---

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)


# --- Train the network ---
print("LEN trainloader:", len(trainloader))

for epoch in range(2): # loop over the dataset multiple times
#for epoch in range(1): # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        #print("LOOP = ", i)
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data
        
        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 10 == 0:
            print(f"[{epoch + 1}, {i + 1:5d}] loss: {running_loss / (i+1):.3f}")
            running_loss = 0.0

print("Finished Training")

# --- Visualize feature map ---
import matplotlib.pyplot as plt

def visualize_feature_map(name):
    fmap = feature_maps[name]           
    fmap = fmap.squeeze(0).cpu().numpy()   # [C, H, W]

    num_channels = fmap.shape[0]
    cols = 8
    rows = (num_channels + cols - 1) // cols

    fig = plt.figure(figsize=(16, 2*rows))
    for i in range(num_channels):
        ax = fig.add_subplot(rows, cols, i+1)
        ax.imshow(fmap[i], cmap='gray')
        ax.axis("off")

    plt.suptitle(f"{name} feature maps")
    plt.show()
    plt.close(fig)


visualize_feature_map("conv1")
visualize_feature_map("conv2")

BATCH SHAPE: torch.Size([4, 1, 32, 32])
Net(
  (conv1): Conv2d(1, 3, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv2): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))
  (fc1): Linear(in_features=150, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=84, bias=True)
  (fc3): Linear(in_features=84, out_features=10, bias=True)
)
LEN trainloader: 378
[1,     1] loss: 2.233
[1,    11] loss: 2.007
[1,    21] loss: 1.005
[1,    31] loss: 0.638
[1,    41] loss: 0.443
[1,    51] loss: 0.313
[1,    61] loss: 0.199
[1,    71] loss: 0.060
[1,    81] loss: 0.003
[1,    91] loss: 0.000
[1,   101] loss: 0.000
[1,   111] loss: 0.000
[1,   121] loss: 0.000
[1,   131] loss: 0.000
[1,   141] loss: 0.000
[1,   151] loss: 0.000
[1,   161] loss: 0.000
[1,   171] loss: 0.000
[1,   181] loss: 0.000
[1,   191] loss: 0.000
[1,   201] loss: 0.000
[1,   211] loss: 0.000
[1,   221] loss: 0.000
[1,   231] loss: 0.000
[1,   241] loss: 0.000
[1,   251] loss: 0.000
[1,   261] loss: 0.000
[1,   271] loss: 0.000
[1,   281] loss: 0.000
[1,   291] loss: 0.000
[1,   301] loss: 0.000
[1,   311] loss: 0.000
[1,   321] loss: 0.000
[1,   331] loss: 0.000
[1,   341] loss: 0.000
[1,   351] loss: 0.000
[1,   361] loss: 0.000
[1,   371] loss: 0.000
Finished Training

Next, I asked chatgpt how I could improve the results. Then I modified the CIFAR-10 classification model so it can be used purely as a feature extractor without labels.

• Do not use the fully connected layers.
• Remove the loss function and optimizer — CrossEntropyLoss is inappropriate because this is not a classification task.
• Do not “train” the model.
• Use the output of the forward pass (after flattening) directly as the feature vector.
• Collect the feature vectors for all images and save them as NumPy arrays in preparation for clustering (e.g., K-means).

# --- No import changes ---

import torch
import torchvision.transforms as transforms
from torch.utils.data import Dataset, DataLoader
from PIL import Image
from pathlib import Path

transform = transforms.Compose([
    transforms.Grayscale(),
    transforms.Resize((32,32)),
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])


class FabAcademyCommitDataset(Dataset):
    def __init__(self, root_dir, transform=None):
        self.root_dir = root_dir
        self.files = list(Path(root_dir).glob("*.jpg"))
        self.transform = transform

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        img = Image.open(self.files[idx])
        if self.transform:
            img = self.transform(img)
        return img, 0   # labelは使わないのでダミー


trainset = FabAcademyCommitDataset(
    root_dir="datasets/commitsImage",
    transform=transform
)

batch_size = 4

trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=False, num_workers=2)

inputs, labels = next(iter(trainloader))
print("BATCH SHAPE:", inputs.shape)

testset = trainset
testloader = trainloader

# --- Define a Convolutional Neural Network ---

import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 3, 5)
        self.pool  = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(3, 6, 5)
        # 全結合層は使わない（特徴抽出器として使う）

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1)  # feature vector
        return x

net = Net()
print(net)

# --- feature map hook  ---
feature_maps = {}

def register_activation_hook(name):
    def hook(module, input, output):
        feature_maps[name] = output.detach()
    return hook

net.conv1.register_forward_hook(register_activation_hook("conv1"))
net.conv2.register_forward_hook(register_activation_hook("conv2"))

# --- No Loss function and optimizer (unsupervised) ---
print("LEN trainloader:", len(trainloader))


# --- Extract features instead of training ---
all_features = []

net.eval()
with torch.no_grad():
    for i, data in enumerate(trainloader):
        inputs, labels = data
        outputs = net(inputs)  # features
        all_features.append(outputs.cpu())

# Concatenate to shape [N, feature_dim]
all_features = torch.cat(all_features, dim=0)
print("Feature shape:", all_features.shape)   # e.g. [1500, 150]


# --- Visualize feature map ---

import matplotlib.pyplot as plt

def visualize_feature_map(name):
    fmap = feature_maps[name]
    fmap = fmap.squeeze(0).cpu().numpy()

    num_channels = fmap.shape[0]
    cols = 8
    rows = (num_channels + cols - 1) // cols

    fig = plt.figure(figsize=(16, 2*rows))
    for i in range(num_channels):
        ax = fig.add_subplot(rows, cols, i+1)
        ax.imshow(fmap[i], cmap='gray')
        ax.axis("off")

    plt.suptitle(f"{name} feature maps")
    plt.show()
    plt.close(fig)


# sample visualization
_ = net(inputs)       # forward to populate hooks
visualize_feature_map("conv1")
visualize_feature_map("conv2")

BATCH SHAPE: torch.Size([4, 1, 32, 32])
Net(
  (conv1): Conv2d(1, 3, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv2): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))
)
LEN trainloader: 378
Feature shape: torch.Size([1509, 150])

Try increasing the image resolution：　28x28 --> 64x64

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

df = pd.read_csv("datasets/fabacademy_commit_weekly_summary.csv")

# Week name
week_cols = [col for col in df.columns if col.startswith("week_")]
weeks = list(range(1, len(week_cols) + 1))  
print(weeks)
#print(commits, commits.dtype)

output_dir = "datasets/commitsImage2"

for idx, row in df.iterrows():
    #if idx >= 3:
    #    break
    commits = row[week_cols].astype(float).values   #cast
    #commits = row[week_cols].values   
    max=commits.max()
    if(max>0):
        commits = commits/max
    #student = row["student_name"]
    #lab = row["lab_name"]
    fig, ax = plt.subplots(figsize=(0.64, 0.64), dpi=100)
    ax.plot(weeks, commits,color="black")
    ax.fill_between(weeks, commits,color="black", alpha=1.0)
    ax.set_facecolor("white")
    ax.axis("off")
    #plt.show()
    plt.savefig(f"{output_dir}/img_{idx}.jpg", dpi=100)
    plt.close(fig)

print("Images are saved")

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]
Images are saved

CNN Unsupervised (unlabeled) learning

import torch
import torchvision.transforms as transforms
from torch.utils.data import Dataset, DataLoader
from PIL import Image
from pathlib import Path

transform = transforms.Compose([
    transforms.Grayscale(),     # 上から実行
    #transforms.Resize((32,32)), # MNISTにあわせたけど、もともと３２でつくりなおそうかな
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])


class FabAcademyCommitDataset(Dataset):
    def __init__(self, root_dir, transform=None):
        self.root_dir = root_dir
        self.files = list(Path(root_dir).glob("*.jpg"))
        self.transform = transform

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        img = Image.open(self.files[idx])
        if self.transform:
            img = self.transform(img)
        return img, 0

trainset = FabAcademyCommitDataset(
    root_dir="datasets/commitsImage2",
    transform=transform
)

batch_size = 4

trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=2) #オリジナルのでよさそう
#trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0)

inputs, labels = next(iter(trainloader))
print("BATCH SHAPE:", inputs.shape)  #1508/4  bactch=4

testset = trainset   # あとでテストをつくる > 来週のために少しとっておくか
testloader = trainloader

# ---　Define a Convolutional Neural Network ---

import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 3, 5)   # 
        self.pool  = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(3, 6, 5)
        self.fc1   = nn.Linear(6 * 13 * 13, 120)
        self.fc2   = nn.Linear(120, 84)
        self.fc3   = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()

print(net)

# --- feature map hook  ---
feature_maps = {}

def register_activation_hook(name):
    def hook(module, input, output):
        feature_maps[name] = output.detach()
    return hook

net.conv1.register_forward_hook(register_activation_hook("conv1"))
net.conv2.register_forward_hook(register_activation_hook("conv2"))


# --- Define a Loss function and optimizer ---

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)


# --- Train the network ---
print("LEN trainloader:", len(trainloader))

for epoch in range(2): # loop over the dataset multiple times
#for epoch in range(1): # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        #print("LOOP = ", i)
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data
        
        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 10 == 0:
            print(f"[{epoch + 1}, {i + 1:5d}] loss: {running_loss / (i+1):.3f}")
            running_loss = 0.0

print("Finished Training")
# --- save model ---
torch.save(net.state_dict(), "fabacademy_cnn.pth")

# --- Visualize feature map ---
import matplotlib.pyplot as plt

def visualize_feature_map(name):
    fmap = feature_maps[name]           
    fmap = fmap.squeeze(0).cpu().numpy()   # [C, H, W]

    num_channels = fmap.shape[0]
    cols = 8
    rows = (num_channels + cols - 1) // cols

    fig = plt.figure(figsize=(16, 2*rows))
    for i in range(num_channels):
        ax = fig.add_subplot(rows, cols, i+1)
        ax.imshow(fmap[i], cmap='gray')
        ax.axis("off")

    plt.suptitle(f"{name} feature maps")
    plt.show()
    plt.close(fig)


visualize_feature_map("conv1")
visualize_feature_map("conv2")

BATCH SHAPE: torch.Size([4, 1, 64, 64])
Net(
  (conv1): Conv2d(1, 3, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv2): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))
  (fc1): Linear(in_features=1014, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=84, bias=True)
  (fc3): Linear(in_features=84, out_features=10, bias=True)
)
LEN trainloader: 378
[1,     1] loss: 2.220
[1,    11] loss: 2.001
[1,    21] loss: 1.009
[1,    31] loss: 0.649
[1,    41] loss: 0.461
[1,    51] loss: 0.343
[1,    61] loss: 0.256
[1,    71] loss: 0.181
[1,    81] loss: 0.080
[1,    91] loss: 0.016
[1,   101] loss: 0.003
[1,   111] loss: 0.000
[1,   121] loss: 0.000
[1,   131] loss: 0.000
[1,   141] loss: 0.000
[1,   151] loss: 0.000
[1,   161] loss: 0.000
[1,   171] loss: 0.000
[1,   181] loss: 0.000
[1,   191] loss: 0.000
[1,   201] loss: 0.000
[1,   211] loss: 0.000
[1,   221] loss: 0.000
[1,   231] loss: 0.000
[1,   241] loss: 0.000
[1,   251] loss: 0.000
[1,   261] loss: 0.000
[1,   271] loss: 0.000
[1,   281] loss: 0.000
[1,   291] loss: 0.000
[1,   301] loss: 0.000
[1,   311] loss: 0.000
[1,   321] loss: 0.000
[1,   331] loss: 0.000
[1,   341] loss: 0.000
[1,   351] loss: 0.000
[1,   361] loss: 0.000
[1,   371] loss: 0.000
[2,     1] loss: 0.000
[2,    11] loss: 0.000
[2,    21] loss: 0.000
[2,    31] loss: 0.000
[2,    41] loss: 0.000
[2,    51] loss: 0.000
[2,    61] loss: 0.000
[2,    71] loss: 0.000
[2,    81] loss: 0.000
[2,    91] loss: 0.000
[2,   101] loss: 0.000
[2,   111] loss: 0.000
[2,   121] loss: 0.000
[2,   131] loss: 0.000
[2,   141] loss: 0.000
[2,   151] loss: 0.000
[2,   161] loss: 0.000
[2,   171] loss: 0.000
[2,   181] loss: 0.000
[2,   191] loss: 0.000
[2,   201] loss: 0.000
[2,   211] loss: 0.000
[2,   221] loss: 0.000
[2,   231] loss: 0.000
[2,   241] loss: 0.000
[2,   251] loss: 0.000
[2,   261] loss: 0.000
[2,   271] loss: 0.000
[2,   281] loss: 0.000
[2,   291] loss: 0.000
[2,   301] loss: 0.000
[2,   311] loss: 0.000
[2,   321] loss: 0.000
[2,   331] loss: 0.000
[2,   341] loss: 0.000
[2,   351] loss: 0.000
[2,   361] loss: 0.000
[2,   371] loss: 0.000
Finished Training

Findings

1. Comparison of MLP and Classical Fitting Methods on Fab Academy Student Data

To compare different fitting approaches on the Fab Academy student count data, I applied several models including polynomial fitting, RBF, nonlinear least squares (tanh), Gaussian, and MLP. Because the dataset is small and the curve has a smooth rise and a gradual decline, several methods behaved differently.

Polynomial Fit

Polynomial fitting is easy to compute. Linear and quadratic fits capture only the rough global trend and fail to represent the gradual decline after the peak. With higher orders (e.g., 10th order), overfitting becomes evident, which is expected given the limited data points.

RBF Fit

Compared with polynomial fitting, the RBF model captured the smoother decline after the peak more effectively. This may be because the chosen centers happened to align well with the shape of the data.

Nonlinear Least Squares (tanh Model)

Although the convergence behavior varied depending on the parameter , the final fitted curves had similar shapes. However, the tanh-based model could not express the gentle decline of the Fab Academy data and tended to produce a step-like shape.

Gaussian Fit

The Gaussian fit remained symmetric and therefore could not model the asymmetric shape of the data — particularly the slow decline after the peak.

MLP

MLPs are powerful because they can approximate arbitrary functions. However, with so few data points, the model behaved similarly to the tanh fit: with 6 hidden units, the output became step-like and failed to reproduce the smooth decline; with 10 hidden units, a peak appeared but the fit showed signs of overfitting.

Conclusion

Despite the limited data, among the methods tested, the RBF fit produced the most reasonable overall approximation to the Fab Academy student count curve, especially in terms of capturing the broad peak and the gradual decline.

2. CNN on Fab Academy Git commits data

In my initial experiment, I transformed the GitLab commit time-series into images and applied a CNN derived from a CIFAR-10 classifier. The resulting feature maps mainly showed simple vertical and horizontal edges, suggesting that the filters were activating but not extracting meaningful structure.

Because my data has no labels and the purpose is feature extraction rather than classification, I redesigned the model: I removed the fully connected layers, removed the loss and optimizer, did not train the network, and instead used the flattened output of the forward pass directly as a feature vector. I then collected these feature vectors for all images and stored them as NumPy arrays for downstream clustering. After increasing the input resolution to 64×64, the extracted features became much more expressive, as seen in the second example.

However, I am still not sure whether these results are correct or sufficient, so I plan to continue studying and refining this approach.