It is optimal to paint them with the most frequent color among the current \(N\) balls.

If \(\text{cnt}[v]\) represents the number of balls with color \(v\) among the \(N\) balls, the answer can be represented as \(\displaystyle N-\max _v\text{cnt}[v]\).

Sample code (Python3)

n = int(input())
c = list(map(int, input().split()))
cnt = [0] * n
for v in c:
    cnt[v - 1] += 1
print(n - max(cnt))