Hmm...\ I'm currently testing it after making my first payment...\ When you need to continuously retrieve API data, if you do it with just one IP,\ you'll defin
Hmm...
I'm currently testing it after making my first payment...
When you need to continuously retrieve API data, if you do it with just one IP,
you'll definitely end up getting blocked for exceeding the request limit.
That's why I decided to use a proxy,
The place I chose was Luminati.
It's more convenient than I expected, and the price isn't bad...
Costs vary from site to site, so you need to check carefully. But this one seems to be somewhere around the middle? It also varies depending on how you parse or crawl, but I think which region's IPs you are assigned is important too.
I chose this service because they say they have a lot of Korean IPs.
However, some API sites connect and others don't, so I had to assign IPs one by one and run tests...
Anyway, the fact is that all the sites I need to connect to right now are accessible without any problems.
You can sign up and use it through the link to the site above.
However, since they charge based on the number of IPs and the amount of traffic,
API crawling probably wouldn't matter much,
but thinking about crawling services that will run the systems I build in the future, I thought they might require more traffic, so I did a bit more searching...
There are also places that charge per number of API Calls (I guess only the more expensive plans are available in Korea...)
Like this one, there are also places that charge per IP and offer unlimited traffic..
(This service is structured by plan. You have to pay $400 to use a minimum of 100 IPs.. The advantage is that there is no traffic limit.. If you need a really large amount of scraping, this service may be a good choice, but I think it's best to choose after calculating how much traffic you'll generate...)
This is said to be the cheapest proxy... It's currently returning a 500 error ( ... )
They don't have Korea, but they look pretty decent... Japan seems to be the closest and best option, but would Japanese IPs keep allowing connections..? (to Korean websites)
For now, I used luminatio.io, but if there's a cheaper service, I'll consider using that one...
The speed isn't bad, and it seems ideal for crawling. ( https://luminati.io/ )
If anyone knows of a good crawling proxy service with reasonable prices and speed that can provide Korean IPs, I'd appreciate it if you let me know in the comments..
