Any large object close to the neighborhood of the sun could be located by reflected sunlight.

Many if not most of the brown dwarfs would appear magenta in color, some orange, others red.

A remarkable property of brown dwarfs is that they are all roughly the same radius as Jupiter. Yes they have greater masses (60–90 Jupiter masses), but their diameter would be about the size of Jupiter.

Why? Lack of fusion to offset the gravitational and electric forces that hold matter together. For brown dwarfs the volume of a brown dwarf is governed primarily by electron-degeneracy pressure.

Coulomb pressure is another factor which means that the radii of brown dwarfs vary by only 10–15% over the range of possible masses. This can make distinguishing them from planets difficult.

While they are not under going fusion (as in what causes the sun to be so bright and hot), they are undergoing gravitational and electronic compression of matter which is leading to them producing higher amounts of various radiation across the spectrum, including infrared (heat) and x-rays and most importantly radio emissions.

What this means to any 'lurking' brown star in our system, it would leave a decided gravitational wake, which would perturb the orbits of planets around the whole system in a manner that would pretty much pinpoint the location of a massive body.

Secondly, they are not black bodies, they do radiate and reflect electromagnet energies across a wide spectrum including light. Yes it is possible to have missed a large body with optics even today in the outer solar system, however the mapping of the whole of space with radio telescopes has been done and continually gets updated all the time.

A near brown dwarf would have signaled its presence via radio waves. And yes there are amateur radio astronomers out there.

Yes over long distances their radio power is too low to pick up more than the faintest of signals. However anything within the solar system is close enough to be detected.

We can still pickup the signals of Voyager craft and their transmissions are infinitesimally small compared to what a modest small brown dwarf can produce.