[1] Fix \(\frac{B+Y}{A+X}\)

Consider minimizing \(X+Y\) for a fixed value of \(k=\frac{B+Y}{A+X}\ (1\leq k)\). Here, we have \(Y=kX+kA-B, X+Y=(k+1)X+kA-B\), and we want \(X\) to be as small as possible. From the condition \(0\leq X,\ Y\), the minimum value that \(X\) can take is \(X=\max(0,\ \lfloor \frac{B-1}{k} \rfloor +1 -A)\).

Thus, the problem can be rephrased into one that asks to find \(k\) that minimizes \((k+1)\max(0,\ \lfloor \frac{B-1}{k} \rfloor +1 -A)+kA-B\).

[2] Use the small number of different values of \(\lfloor \frac{B-1}{k} \rfloor\)

For values of \(k\) with the same value of \(\lfloor \frac{B-1}{k} \rfloor\), the value of the above formula will be smaller when \(k\) is smaller. Thus, for every value \(Q\) that \(\lfloor \frac{B-1}{k} \rfloor\) can take, let us find the smallest \(k\) such that \(\lfloor \frac{B-1}{k} \rfloor =Q\) and compute the value of the formula, and then find the minimum of those values. When \(1 \leq Q\), the smallest value of \(k\) for \(Q\) can be found as \(\lfloor \frac{B-1}{Q+1} \rfloor + 1\), which can be computed in \(O(1)\) time, from the following:

\(Q=\lfloor \frac{B-1}{k} \rfloor \iff Q \leq \frac{B-1}{k} < Q+1 \iff \lfloor \frac{B-1}{Q+1} \rfloor + 1\leq k \leq \frac{B-1}{Q}\).

For \(Q=0\), the smallest value of \(k\) is \(B\).

Let us list the values that \(Q\) can take. First, we have \(\lfloor \frac{B-1}{k}\rfloor\) for \(k=1,\ 2,\ \dots,\ \lceil \sqrt{B-1}\rceil\). Next, we have \(\lfloor \frac{B-1}{k}\rfloor\) for \(\lceil \sqrt{B-1}\rceil < k\), where the values are clearly at most \(\lceil \sqrt{B-1} \rceil\), which means there are only \(O(\sqrt{B})\) of them.

From the above, the problem can be solved in \(O(\sqrt{B})\) time per test case.