You're right, you're missing other ways to get new artifacts. According to Stardew Valley Wiki, artifacts can be acquired in the following ways:
- Using the Hoe on worm tiles. Different parts of the map will drop different artifacts. These locations are much more common in winter.
- Using a Hoe on dirt located within The Mines and Skull Cavern.
- Killing Monsters.
- Fishing for treasure chests.
In case you want a list with the artifacts and their locations, see below or refer to this page:
Dwarf Scroll I
Mines level 1-105 (maybe every level, not confirmed)
Green Slime (Secret Woods)
Dwarf Scroll II
Mines level 40-65
Blue Slime (Secret Woods)
Dwarf Scroll III
Mines level 40+
Lava Bat (Mines Level 81+)
Blue Slime (Secret Woods)
Skull Cavern
Dwarf Scroll IV
Any Monster (Mines)
Using hoe on dirt (Mines level 81+)
Chipped Amphora
Town (4%)
Arrowhead
Mountain (2%)
Forest (2%)
Bus Stop (2%)
Ancient Doll
Mountain (4%)
Forest (3%)
Bus Stop (3%)
Town (1%)
Fishing Chest (Ocean/Secret Woods)
Elvish Jewelry
Forest (1%)
Fishing Chest (?%)
Chewing Stick
Mountain (2%)
Forest (2%)
Town (1%)
Fishing Chest (?%)
Ornamental Fan
Beach (2%)
Forest (1%)
Town (0.8%)
Fishing Chest (River)
Dinosaur Egg
Mine (1%)
Mountain (0.8%)
Fishing chest (Ocean/River)
Rare Disc
Underground Mine (1%)
Fishing Chest (Ocean/River)
Ancient Sword
Forest (1%)
Mountain (0.8%)
Fishing Chest (Ocean/River)
Rusty Spoon
Town (5%)
Fishing Chest (River ?%)
Rusty Spur
Farm (10%)
Fishing Chest (River ?%)
Rusty Cog
Mountain (5%)
Mine ~14 in dirt
Fishing Chest (Ocean/River)
Chicken Statue
Farm (10%)
Fishing Chest (?%)
Ancient Seed
Forest (1%)
Mountain (1%)
Fishing Chest (River)
Prehistoric Tool
Bus Stop (4%)
Mountain (3%)
Forest (3%)
Fishing Chest (?%)
Dried Starfish
Beach (10%)
Fishing Chest (River ?%)
Anchor
Beach (5%)
Fishing Chest (?%)
Glass Shards
Beach (10%)
Fishing Chest (Ocean)
Bone Flute
Underground Mine (2%)
Mountain (1%)
Forest (1%)
Town (0.5%)
Fishing Chest (River)
Prehistoric Handaxe
Mountain (5%)
Forest (5%)
Bus Stop (5%)
Dwarvish Helm
Underground Mine (1%)
Dwarf Gadget
Underground Mine (0.1%)
Ancient Drum
Underground Mine (2%)
Bus Stop (1%)
Forest (1%)
Town (0.5%)
Golden Mask
Desert (4%)
Golden Relic
Desert (8%)
Strange Doll 1
Everywhere (0.1%)
Fishing Chest (?%)
Strange Doll 2
Everywhere (0.1%)
Fishing Chest (?%)
Prehistoric Scapula
Town (1%)
Forest (?%)
Prehistoric Tibia
Forest (1%)
Railroad (?%)
Prehistoric Skull
Mountain (1%)
Skeletal Hand
Beach (1%)
Prehistoric Rib
Town (1%)
Prehistoric Vertebra
Bus Stop (1%)
Skeletal Tail
Underground Mine (1%)
Fishing Chest
Nautilus Fossil
Beach (3%)
Fishing Chest (Ocean/River)
Amphibian Fossil
Forest (1%)
Mountain (1%)
Fishing Chest (Ocean)
Palm Fossil
Forest (1%)
Beach (1%)
Desert (1%)
Trilobite
Forest (3%)
Beach (3%)
Mountain (3%)
Best Answer
There is actually a benefit from perfect catches, but it's not obvious ingame. In addition to extra star tokens in the Stardew Fair, it also multiplies the skill experience received from catching any fish by 1.4.
As of version 1.4, a perfect catch additionally increases the quality of silver and gold star fish by one level.