Baptism is a sign and seal of the covenant of grace, and of the cleansing away of sins by union with Christ (Mark 1:4; Colossians 2:11, 12; Galatians 3:27). This sacred ordinance is performed in the name of the Father, and of the Son, and of the Holy Spirit (Matthew 3:11; 28:19; John 1:33), and is rightly administered by sprinkling or pouring water on the applicant (Ezekiel 36:25-27; Hebrews 9:10, 19-22; Acts 2:41; 16:33; Mark 7:4). [Note that in Hebrews 9:10 washings is literally baptisms in the Greek, and that in the original Greek of Mark 7:4 the tables or couches undergoing the Old Testament ceremonial washing by sprinkling (cf. Numbers 8:7, 19:18) are literally said to be baptized, thus destroying the notion that the Greek word baptizo refers to dipping rather than ceremonial cleansing in the New Testament.]

The Old Testament sign of the covenant of grace was circumcision, which, like baptism, represents the taking away of sin by union with the crucified and risen Christ: “In whom also ye are circumcised with the circumcision made without hands, in putting off the body of the sins of the flesh by the circumcision of Christ: Buried with him in baptism, wherein also ye are risen with him through the faith of the operation of God, who hath raised him from the dead. And you, being dead in your sins and the uncircumcision of your flesh, hath he quickened together with him, having forgiven you all trespasses” (Colossians 2:11-13; compare with Matthew 3:11, 12).

Again like baptism, circumcision was to be received upon a profession of faith: “And he [i.e., Abraham] received the sign of circumcision, a seal of the righteousness of the faith which he had yet being uncircumcised: that he might be the father of all them that believe, though they be not circumcised; that righteousness might be imputed unto them also” (Romans 4:11). Compare with Acts 8:12: “But when they believed Philip preaching the things concerning the kingdom of God, and the name of Jesus Christ, they were baptized, both men and women.”

Although the Old Testament sign of the covenant was, in adults, to be administered after they had believed on the Messiah (whose righteousness was imputed to them), the sign was given not only to the heads of households who believed (Genesis 15:6) but also to their male children, who were to be brought up in the nurture and admonition of the Lord (Genesis 18:19; Ephesians 6:4): “And I will establish my covenant between me and thee and thy seed after thee in their generations for an everlasting covenant, to be a God unto thee, and to thy seed after thee . . . This is my covenant, which ye shall keep, between me and you and thy seed after thee; Every man child among you shall be circumcised . . . He that is born in thy house, and he that is bought with thy money, must needs be circumcised: and my covenant shall be in your flesh for an everlasting covenant” (Genesis 17:7, 10, 13).

That the children of one or both believing parents are to be considered holy and to receive the sign of the cleansing away of sin by the blood of Christ is also clear from such New Testament passages as 1 Corinthians 7:14: “For the unbelieving husband is sanctified by the wife, and the unbelieving wife is sanctified by the husband: else were your children unclean; but now are they holy.”

The apostle Peter calls on sinners to repent and to be baptized, and proceeds to explain that the promise is not only to them but also to their children and to whomever else the Lord is pleased to call from darkness to light: “Then Peter said unto them, Repent, and be baptized every one of you in the name of Jesus Christ for the remission of sins, and ye shall receive the gift of the Holy Ghost. For the promise is unto you, and to your children, and to all that are afar off, even as many as the LORD our God shall call” (Acts 2:38, 39).

Therefore, it was the practice of the apostles from the very beginning to baptize households, just as circumcision was administered to households under the Old Testament dispensation of the covenant of grace: “And he took them the same hour of the night, and washed their stripes; and was baptized, he and all his, straightway” (Acts 16:33); “And I baptized also the household of Stephanas: besides, I know not whether I baptized any other” (1 Corinthians 1:16).

The blessings of the covenant of grace are not diminished in the New Testament era, nor is the sign and seal removed from any; rather, the blessings are increased and extended to the Gentile nations (Isaiah 52:15; Acts 2:39; Galatians 3:9, 14), and the sign extended to include believing women and their children: “And when she was baptized, and her household, she besought us, saying, If ye have judged me to be faithful to the Lord, come into my house, and abide there. And she constrained us” (Acts 16:15).

It is possible for the children of believers to be partakers of saving grace while still incapable of hearing the Gospel – "he shall be filled with the Holy Ghost, even from his mother's womb" (Luke 1:15, cf. 41, 44) – and when such children die as infants they are received into glory, as are all others who are savingly united with Christ (2 Samuel 12:23).

We see then that the infant children of believers, who may themselves receive the Spirit of Christ and who will be brought up in the nurture and admonition of the Lord, are to be included in the same covenant as their parents – rather than being counted as under the covenant of works, as are the heathen and their children (Jeremiah 10:25). Although believers’ children must not be considered as saved until they show signs of the new birth, the promise is to them through their parents, who must believe and take hold of the promises for their children. "And they said, Believe on the Lord Jesus Christ, and thou shalt be saved, and thy house" (Acts 16:31). It is because of their parents that infant children are outwardly sanctified. Thus it is not pleasing to God to deny to such infants the sign and seal of union with the Saviour and of cleansing by His blood and Spirit.

And they brought young children to him, that he should touch them: and his disciples rebuked those that brought them. But when Jesus saw it, he was much displeased, and said unto them, Suffer the little children to come unto me, and forbid them not: for of such is the kingdom of God. Verily I say unto you, Whosoever shall not receive the kingdom of God as a little child, he shall not enter therein. And he took them up in his arms, put his hands upon them, and blessed them (Mark 10:13-16)