The dagger key is an item obtained during the Broken Home quest. It can be found on the floor in one of the cells on the bottom-most floor. It is used to unlock the Dagger Door within the mansion, which leads to Ingram's room.
The fact that it can be found that deep in the ancient ruins indicates both that Ingram was quite thorough in his exploration of the area, and that he brought the key with him and lost it during the journey.
