The Wagener house is a two story, three by three bay house, rectangular in plan and covered by a gable roof. A pedimented cornice, broad frieze and lunette highlight the roof line. The house is constructed of stone, which has been parged. A small one story, one by one bay, gable roofed wing extends from the west elevation of the dwelling. A one story portico extends from the east elevation. Windows throughout the dwelling feature stone sills and lintels and contain twelve-over-twelve light sash. Aluminum triple track storm windows were installed in the late 1960's. Some the original shutters are stored in the attic.

A monumental, two story, classical portico of Roman Doric columns, molded entablature and full pediment dominate the formal, south elevation. The first floor of this elevation features a pair of entrances composed of French doors. The second story features a balcony, which spans the full width of the elevation. A central entrance leads out to the balcony. This entrance is flanked on each side by a single window.

19th Century photos show that the original columns were rectangular. Morris replaced these columns in 1908. After suffering from years of decay, the wooden column bases were replaced in the early 1970's. The combination of the exposed location, little maintenance, and nesting of Starling's inside the columns resulted in extensive rot and decay. The columns, bases, and capitals were completely replaced in a 1996 renovation of the portico. The 24” diameter Roman Doric columns were made in Oregon from 8/4 clear all-heart redwood, treated with wood preservative, and coated with asphalt on the interior to resist decay. The bases and capitals are molded polyurethane. The capitals were flashed with copper. Ventilation in the bases and attic serve to minimize the accumulation of condensation. Other renovations at this time included complete replacement of the east and west masonry piers, replacement of the porch framing, installation of mahogany decking, removal of the west stairs, and complete rehabilitation of the railings.

The east elevation of the dwelling features three evenly spaced windows on the second story and two evenly spaced windows on the first story. The principle entrance to the dwelling is located in the northernmost bay of this facade. The entrance, which serves as the formal entrance to the dwelling, is composed of a six panel door encased by an elaborate surround, which is surmounted by an eight light transom. The entrance is sheltered by a ca.1908 portico composed of a gable roof supported by a full entablature and paired Tuscan columns. The porch was extended during the Morris renovations. A National Register of Historic Places plaque was placed here in 1995.

The north (rear) elevation of the dwelling features two, evenly spaced windows on the first, second and attic stories. A modern casement window has been installed between the two windows on the second floor of this elevation.

The west elevation of the house is dominated by a single story, one by one bay, frame wing, which is covered by a pedimented gable roof. A sheltered entrance is located on the north elevation of this wing. A stair, on this elevation, that leads from the side yard up to the floor of the south portico was removed during the 1996 portico reconstruction.

Arcade - Hanging in the Oliver House Museum in Penn Yan is an oil on linen painting entitled “The Wagener Mansion on Bluff Point”. On the back there is a faint pencil inscription which reads “Painted by Miss Ida Wagener Penn Yan June 22, 1858 Yates Co.” This painting shows an arcade consisting of four arches extending to the west of the house. There appears to be a chimney structure at the west end. It is unclear if this arcade actually existed or was rather artistic license.