The answer is all of these.
The Duke is the speaker of the poem where he is entertaining an emissary who has come to negotiate the Duke's marriage. As he shows the visitor through his palace, they came across the portrait of the late Duchess. The Duke then began to reminisce about the portrait sessions and then about the Duchess herself. As his monologue continues, the reader realizes that it was the Duke who caused the untimely demise of the previous Duchess.
