St. Mary's Parish was established as a community of the faithful in 1832. It is the oldest Catholic parish in New Haven and the second oldest in Connecticut.

In 1832 Father James McDermot, a priest of the Diocese of Boston, which encompassed all of New England, was sent to assist the pastor of Holy Trinity Church in Hartford. Fr. McDermot's special duty was ministry to the Catholics in the western part of the State of Connecticut including New Haven, Bridgeport, Norwalk, Derby, Waterbury, Meriden, and Middletown. Later that year he became the resident pastor in New Haven while continuing the pastoral care of the Catholics in the western Connecticut. At that time, there were about two hundred Catholics in New Haven.

Land was purchased near the intersection of York Street and Milford Turnpike Road in New Haven. In 1834 a church was completed on this site and given the name "Christ's Church." Fr. McDermot was succeeded for a few months by Fr. William Wiley and then by Fr. James Smyth. In Father Smyth's last year as pastor in 1848, the Church was destroyed by fire. Several months after the fire the property occupied by the Church Street Congregation Society was made available. On December 16, 1848, Fr. Philip O'Reilly, who succeeded Fr. Smyth as pastor, took title to the property and the Congregational Meeting House became the Catholic Church in New Haven. Bishop Tyler of Hartford dedicated the Church to St. Mary.

Over the next twenty years the ambitious and impressive edifice that is the present church developed - at a time when the city's Catholic population was steadily increasing. Father O'Brien was appointed pastor in 1851, succeeding Fr. O'Reilly. It was in 1867 that the Dominican Friars came and preached a parish mission that was long remembered. During the mission, it became evident that the parish needed a larger Church. Fr. O'Brien developed plans for this Church to be the most beautiful large Gothic church in Connecticut. On June 13, 1868, the parish took title to a parcel of land between Hillhouse Avenue and Temple Street. The cornerstone for the present Church was laid on September 22, 1870. Fr. Murphy succeeded Fr. O'Brien, becoming pastor in May 1872. He brought the construction of the Church to completion and it was dedicated on October 25, 1874. Built on New Haven's finest residential street, Hillhouse Avenue, St. Mary's represented the faith, the pride, and the high hopes of New Haven's struggling Catholic families.

The next pastor of St. Mary's was Fr. Patrick Lawlor, appointed in 1879. During his years as pastor, excellent curates ably assisted Father Lawlor. One of these was Fr. Michael McGivney. On October 2, 1882, in the basement of St. Mary's Church, Father Michael J. McGivney organized the Knights of Columbus as a fraternal benefit society that would protect the widows and children of working men and foster their faith and their social progress. Fr. McGivney spent seven years in service to the parish and in 1884 was appointed pastor of St. Thomas Church in Thomaston, CT. In August 1890 at the age of 38, he died in Thomaston, CT after a struggle with pneumonia and was buried in Waterbury, CT.

On Sunday, May 16, 1886 Dominican Fathers Higgins, Kent, Hartigan and Linahan were charge with the ministry at St. Mary's and on that day the first Solemn Mass in the Dominican Rite was celebrated in the Church. It was made clear that Bishop McMahon expected "great results" from this foundation of the Dominicans in his diocese.

The friars came to a parish that was geographically very large and one that had been well nurtured by their predecessors. Large number of Irish immigrants had come to the New Haven area and greater numbers continued to arrive each day. The parish was large, young, active and dedicated to the faith and to the Church.

The educational needs of a parish with hundreds of young families were soon to be met with the arrival of the Dominican Sisters of St. Mary's of the Springs in 1901. The Sisters established a convent and an academy at 440 Orange Street. At its inception it was both a grammar school and a high school. In September 1904 the parochial school conducted by the Sisters was opened on Ashmum Street. A new grammar school and convent on Prospect Street was purchased in 1945. The parochial school continued to serve the parish until the mid 1980s.

The changes brought by the influx of immigrants were not the last ones for the Dominicans and the Parish of St. Mary. Until the early 1960s St. Mary's Parish was a flourishing residential parish. The appeal of the suburbs drew many parishioners away from the city and St. Mary's. As a result, St. Mary's took on a different character.

Today St. Mary's provides a home for a widely diverse congregation. Many are lifetime members of St. Mary's, the descendents of the founding families and early immigrants. Others have joined the parish more recently, and still others come to the community for only a short time to purse academic programs or temporary employment in the area. Single people and families from New Haven, Hamden, West Haven, East Haven, as well as surrounding towns join together to worship in liturgy and to receive the preached word at St. Mary's, as it continues to be the primary Catholic presence in downtown New Haven.