Over the years the idea developed that if you give some of what you own from which you can make a living to God, this expresses the value of your relationship with God. Giving to God by burning it was also a idea that contributed to the idea of sacrificing to God.
It is notable that Cain gave "some" fruits of the ground, but Abel gave "the firstlings of his flock, including their fat." Genesis 4:3,4
By these descriptions, Abel had given his best in sacrifice, while Cain just gave some of what he had. God knew their what was in their hearts, their inclinations, and what these men gave to God could well have been an indication of how they felt about God. Abel felt that his God deserved his best, his first and fattest from his flock, but Cain apparently did not, he gave up "some".
Much later, when animal sacrifice became a command for the nation of Israel, it wasn't just any animals that were required. It had to a sound animal, with no defect, otherwise, its sacrifice would gain no approval from God. Leviticus 22:18-20; Deuteronomy 17:1
About a thousand years later God inspired Malachi to tell the unfaithful priests of his day that when they gave or accepted a blind, stolen, lame or sick animal for sacrifice, then they were despising his name, and dishonoring him. He even pointed out that if those priests gave that kind of gift to a human governor, would the governor accept it or treat them kindly? Malachi 2:6-15
So it is perceived value that was the issue here. People did not use gold or silver coins until later, but from the very beginning, with Cain and Abel, the food that they raised, or the number of healthy animals that they possessed, or even the amount of land that they owned were among the things that had value in their eyes. Both Cain and Abel understood this, so when the time came for them to express the value of their relationship with God, they used these tangible things to do so, and gave them to God.
Another issue is faith. Later on Abel was commended in God's word as a person of faith, and that faith gave a greater worth to his sacrifice:
“By faith Abel offered God a sacrifice of greater worth than Cain, through which faith he had witness borne to him that he was righteous, God bearing witness respecting his gifts.” (Hebrews 11:4)
On the other hand, Cain showed a lack of faith by his lack of love for his brother, and because he had wicked works.
1 John 3:10-12
"For this is the message that you have heard from the beginning, that we should love one another; 12 not like Cain, who originated with the wicked one and slaughtered his brother. And for what reason did he slaughter him? Because his own works were wicked, but those of his brother were righteous."
Knowing that, God had it written that "While Jehovah looked with favor on Abel and on his offering, he did not look with any favor on Cain and on his offering." Genesis 4:4-5