Below working code has suddenly started failing with error
HTTP error fetching URL. Status=403, URL=[https://www.valueresearchonline.com/funds/26123/motilal-oswal-flexi-cap-fund-regular-plan]
I have got below Cookie value by logging into https://www.valueresearchonline.com/ with ( [email protected] / 0987654321 )
I'm not understanding what is broken suddenly ?
Jsoup.connect("https://www.valueresearchonline.com/funds/26123/motilal-oswal-flexi-cap-fund-regular-plan")
.timeout(15000)
.userAgent("Mozilla")
.header("Cookie", "PHPSESSID=6d9v48p1i5lpgm7pi75ag0okvq; currency=INR; magnitude=LC; ad=ee5ceff9a39de83a4dfcd9cc96efd7aa04912966; ad=ee5ceff9a39de83a4dfcd9cc96efd7aa04912966; wec=296642702; nobtlgn=714443298; ac=68156306%7C526294102%7C424761678; ac=68156306%7C526294102%7C424761678; _gcl_au=1.1.443352322.1663577511; _gid=GA1.2.307799405.1663577512; _fbp=fb.1.1663577512204.684634655; _clck=fgstbv|1|f50|0; __gads=ID=5ed6c786ebad7ee2-2263761e9bd600b6:T=1663577512:S=ALNI_Mb0p_Gif2EChNCORy7JOdTU7x4kjA; __gpi=UID=000009ce9ba54907:T=1663577512:RT=1663577512:S=ALNI_MbPL5xoHxgY76gU9mzDzJltULL80Q; __cf_bm=iIVI9aabT4vAdAmvQQzQTDDs9z4MPaMB1gv602Vn2rI-1663577514-0-AQ9ZKhXneLwVKm6CKEzLoY2EKcrIlNB82wgEPDw7taV6k/fnqTzp0L5zrpAl0fnkF1dn7Ac1DyNdfOnsgCTjBZx5Y6ia4Pvj2ceyIBfyXcIYpR8JkYTYGHfqPlrncv7k6Q==; alp=VROL; aa=364476%7C230264168%7C651860858; aa=364476%7C230264168%7C651860858; arl=604590238; arl=604590238; PERMA-ALERT=0; pgv=6; _ga=GA1.1.1410692956.1663577512; _ga_N9R425YFBJ=GS1.1.1663577511.1.1.1663577540.31.0.0; _clsk=1prm412|1663577540567|4|1|l.clarity.ms/collect")
.method(Connection.Method.GET)
.execute();
Example 2 : Failing
Connection.Response initial1 = Jsoup.connect("https://www.valueresearchonline.com/login/?target=%2fmyaccount")
.timeout(60000)
.userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:49.0) Gecko/20100101 Firefox/49.0")
.data("username", "[email protected]")
.data("password", "0987654321")
.method(Connection.Method.POST)
.execute();
System.out.println("Call 1.1 " + initial1.statusCode());
System.out.println(initial1.cookies().values());
Adding login debug console from chrome
UPDATE
I could get this working with hard coded value of cookies from chrome console but I'm not sure how to get value of cookies dynamically ?
Please check
STEP 1 ( hard coded cookie value )
String apiUrl = "https://www.valueresearchonline.com/api/check-user/";
// Connect to the API URL
Connection.Response response1 = Jsoup.connect(apiUrl)
.method(Connection.Method.GET)
.timeout(60000)
.header("User-Agent", "Mozilla/5.0")
.header("Accept", "application/json")
//Hard Coded, taken from Chrome after valid email is entered.
//Can we get this cookie value dynamically or runtime ??
.header("Cookie", "HARD_CODED_COOKIE_VALUE_FROM_CHRIME")
.data("q", "[email protected]")
.data("password", "1")
.ignoreContentType(true) // Ignore content type to parse non-HTML response
.execute();
// Get the cookies from the response
Map<String, String> cookies1 = response1.cookies();
STEP 2 - Cookie value generated from STEP 1
Connection.Response response2 = Jsoup.connect("https://www.valueresearchonline.com/login/?target=%2f%3f&utm_source=home&utm_medium=vro&utm_campaign=desktop-profile-menu")
.timeout(60000)
.userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:49.0) Gecko/20100101 Firefox/49.0")
.header("Cookie", getCookiesString(cookies1))
.header("Host", "www.valueresearchonline.com")
.data("username", "[email protected]")
.data("password", "0987654321")
.method(Connection.Method.POST)
.ignoreContentType(true)
.execute();
How can I get STEP 1 cookies value without hardcoding ?