newrelic agent is not sending data to newrelic servers at staging only

Question

I have a new relic agent configured like so:

newrelic==2.56.0.42

    newrelic.agent.initialize(newrelic_ini_file, newrelic_env)
    logging.info('NewRelic initialized with newrelic_env '+repr(newrelic_env))
    logging.info('NewRelic config name is '+repr(newrelic.agent.application().name))

It's logging the "right" things in staging, but it's not sending.

 NewRelic initialized with newrelic_env 'staging'
 NewRelic config name is 'My Service (Staging)'

The agents send data from localhost and production.

This is some configs from the newrelic.ini file

[newrelic]
license_key = xxxxx
app_name = My Service
monitor_mode = true
log_file = /tmp/newrelic-python-agent.log
log_level = info

This is my staging config in newrelic.ini

[newrelic:staging]
app_name = My Service (Staging)
monitor_mode = true
log_level = debug

One log that I think is suspicious is this

(14/NR-Harvest-Thread) newrelic.core.agent DEBUG - Completed harvest of all application data in 0.00 seconds.

I think the agent not collecting any data due to the 0.00 seconds part.

What debug logs should I look for, for validating actual data being sent and received 200 from new relic.

Additional details: staging is running on Docker alpine:3.6

Kuroshio

tried debugging tool of new relic? docs.newrelic.com/docs/agents/manage-apm-agents/troubleshooting/… – Cantoris

@Cantoris I did. all success in all items except for log Failure Base/Log/Copy and no explanation on that error – Kuroshio

Add log_File param in newrelic config and verbose:1 . to get detailed logs – Cantoris

Answer 1

The clue to the answer was indeed the debug log of harvest of all application data in 0.00 seconds.

apparently, newrelic.agent.initialize has to go BEFORE app = Flask(__name__). Moved before and it started sending.

Kuroshio

Answer 2

According to newrelic discuss, the Failure Base/Log/Copy can be resolved by configuration change

You can include this line in your config file, as well, which will stop them from going to the default location:

log_to_stdout: false

Another issue can be that log level should be info

log_level = info

Logging at debug can generate a lot of data very quickly. Monitor the size of your log file closely, changing log_level back to info as you finish troubleshooting.

EDIT

Issue was asked in newrelic discuss.

Ulberto

1. I don't think this Failure is related to newrlic reporting since it's in staging env only. 2. log_to_stdout: false didn't solve the Failure Base/Log/Copy error – Kuroshio

@Kuroshio do you have different license_key in stage? – Ulberto

It used to be info and still didn't work. updated my questions with configs in the general section (not specific to staging) – Kuroshio

@Kuroshio can you add –enable-threads ? see ianozsvald.com/2014/12/10/… – Ulberto

last resort, see NewRelic answer with similar issue discuss.newrelic.com/t/… – Ulberto

Let us continue this discussion in chat. – Kuroshio