The secret Goblin temple is located in the north part of the Goblin cave. The stairs leading to it are guarded by 2 goblin guards, who will only allow you to pass if you are in goblin form after drinking a goblin potion. The temple is heavily involved in the Land of the Goblins quest, as well as the Hopespear's Will miniquest that follows it. The temple was originally built overground, on the Plain of Mud, but it sank underground.
Inside the temple there is an altar, which is available to recharge Prayer after completing the quest Land of the Goblins. Six goblin tribes have chambers located around the temple with entry only if wearing the proper coloured goblin mail. None of the goblins within the temple can be attacked.
If the you turn back to human form whilst in the temple, for example by equipping non-goblin items, then you will be immediately kicked out of the temple and will be returned to its entrance. You can however turn back to human form in the crypt at the north of the temple, since there are no goblins in here.
Land of the Goblins
During the Land of the Goblins, you must travel with Zanik to the temple in order to find the location of Yu'biusk. Zanik is allowed into the temple but later captured, you must use a Goblin potion in order to enter. By talking to the High Priest you learn that you must talk to Snothead in the crypt to the north of the temple. In order to gain access to the crypt, you must colour your goblin mail to the colours of each tribe and pickpocketing keys from their priests. Once the keys have been stolen, you can enter the crypt. You are required to defeat each of the five spirits in the crypt, after which you will be revealed the location of Yu'biusk.
Hopespear's Will miniquest
During this miniquest you must once again defeat each of the spirits within the crypt, but this time without weapons or armour. Once this has been done, you must take their bones to Yu'biusk to bury them.