Checking Vmware Perf Logs

login as: root

VMware vCenter Server Appliance 6.0.0.20000

Type: vCenter Server with an external Platform Services Controller

root@10.1.7.20's password:
Last login: Mon Jun 13 16:03:32 2016 from pbtpvmpxe.st.com
Connected to service

    * List APIs: "help api list"
    * List Plugins: "help pi list"
    * Enable BASH access: "shell.set --enabled True"
    * Launch BASH: "shell"

Command>
Command> shell.set --enabled True
Command> shell
    ---------- !!!! WARNING WARNING WARNING !!!! ----------

Your use of "pi shell" has been logged!

The "pi shell" is intended for advanced troubleshooting operations and while
supported in this release, is a deprecated interface, and may be removed in a
future version of the product.  For alternative commands, exit the "pi shell"
and run the "help" command.

The "pi shell" command launches a root bash shell.  Commands within the shell
are not audited, and improper use of this command can severely harm the
system.

Help us improve the product!  If your scenario requires "pi shell," please
submit a Service Request, or post your scenario to the
https://communities.vmware.com/community/vmtn/vcenter/vc forum and add
"appliance" tag.

p111vc01:~ # cd /var/log/vmware/
p111vc01:/var/log/vmware # ll
total 136
drwxr-xr-x 3 root           root       4096 Jun 13 16:02 applmgmt
drwxr-xr-x 2 root           root       4096 Jun 12 22:10 cloudvm
drwxr-xr-x 4 cm             cis        4096 Jun  3 22:50 cm
drwxr-xr-x 3 eam            cis        4096 Jun 13 12:24 eam
drwx------ 9 root           root       4096 Jun 12 08:42 esx
drwx------ 2 root           root       4096 Jan 21 07:10 iiad
drw-r--r-- 2 root           root       4096 Jun 13 15:40 invsvc
lrwxrwxrwx 1 root           root         19 Mar 17 21:50 journal -> /storage/db/journal
drwxr--r-- 2 mbcs           cis        4096 Feb 12 11:53 mbcs
drwxrwx--- 2 netdumper      netdumper  4096 Mar 17 23:43 netdumper
drwxr--r-- 2 perfcharts     cis       20480 Jun 13 11:12 perfcharts
drwxrwxr-x 2 deploy         deploy     4096 Jun 12 23:56 rbd
drwx------ 3 root           root       4096 Jun  3 22:50 rhttpproxy
drwx------ 2 root           root       4096 Oct 13  2015 rsyslogd
drwxr-xr-x 3 root           root       4096 Jun 13 15:32 sca
drwxr-xr-x 3 root           root       4096 Jun 12 07:41 syslog
dr-xr-x--- 3 vapiEndpoint   cis        4096 Jan 14 02:49 vapi
dr-xr-x--- 2 vapiEndpoint   cis        4096 Jan 14 02:49 vapi-endpoint
drwxr-xr-x 2 root           root       4096 Mar  8 05:22 vctop
drwxr-xr-x 3 vdcs           root       4096 Jun 13 01:01 vdcs
drwxr-xr-x 2 root           root       4096 Oct 13  2015 vmafd
drwx------ 2 root           root       4096 Jun 13 03:00 vmafdd
drwxr-xr-x 2 root           root       4096 Jan 13 01:30 vmdir
drwxr-xr-x 2 root           root       4096 Jun 13 02:15 vmware-sps
drwxr-xr-x 3 vpostgres      root       4096 Nov 11  2015 vpostgres
drwxr-xr-x 6 root           root       4096 Jun 13 14:06 vpxd
drwxr-xr-x 2 vsan-health    root       4096 Jun 13 02:27 vsan-health
drwxr-xr-x 3 vsm            cis        4096 Jun  9 05:31 vsm
drwxr-xr-x 4 vsphere-client root       4096 Jun  3 22:52 vsphere-client
drwxr-x--x 3 root           root       4096 Oct 13  2015 vws
drwxr-xr-x 2 vpx-workflow   cis        4096 Jun  3 22:52 workflow
p111vc01:/var/log/vmware # cd vsphere-client/
p111vc01:/var/log/vmware/vsphere-client # ll
total 14116
drwx------ 5 vsphere-client users    4096 Mar  4 16:00 logbrowser
drwxr-xr-x 3 vsphere-client root     4096 Jun  6 13:57 logs
-rw------- 1 vsphere-client users 1049246 Apr 24 16:35 vsphere-client-gc.log.0
-rw------- 1 vsphere-client users  913668 Jun 13 16:03 vsphere-client-gc.log.0.current
-rw------- 1 vsphere-client users 1048908 Apr 26 03:16 vsphere-client-gc.log.1
-rwx------ 1 vsphere-client users 1048703 May  4 07:22 vsphere-client-gc.log.2
-rwx------ 1 vsphere-client users 1049110 May 23 11:40 vsphere-client-gc.log.3
-rw------- 1 vsphere-client users  627862 Jun  3 22:30 vsphere-client-gc.log.4.current
-rwx------ 1 vsphere-client users 1049207 Feb  9 06:06 vsphere-client-gc.log.5
-rwx------ 1 vsphere-client users 1048998 Feb  9 10:59 vsphere-client-gc.log.6
-rwx------ 1 vsphere-client users 1048889 Feb  9 17:25 vsphere-client-gc.log.7
-rwx------ 1 vsphere-client users 1049187 Feb  9 23:21 vsphere-client-gc.log.8
-rwx------ 1 vsphere-client users 1049262 Feb 10 04:54 vsphere-client-gc.log.9
-rwx------ 1 vsphere-client users 3372895 Jun  4 10:40 wrapper.log
p111vc01:/var/log/vmware/vsphere-client # cd logs/
p111vc01:/var/log/vmware/vsphere-client/logs # ll
total 112512
drwx------ 2 vsphere-client users    20480 Jun 13 00:00 access
-rwx------ 1 vsphere-client users  2138308 Jun  4 10:40 eventlog.log
-rwx------ 1 vsphere-client users        0 Oct 13  2015 query_profile_dataservice.log
-rw------- 1 vsphere-client users 51813791 Jun 13 16:04 vsphere_client_virgo.log
-rw------- 1 vsphere-client users  2464272 Jun  6 13:57 vsphere_client_virgo_0.log.zip
-rw------- 1 vsphere-client users  2412590 May 31 02:54 vsphere_client_virgo_1.log.zip
-rw------- 1 vsphere-client users  2360444 May 24 10:33 vsphere_client_virgo_2.log.zip
-rw------- 1 vsphere-client users  2348343 May 17 21:07 vsphere_client_virgo_3.log.zip
-rw------- 1 vsphere-client users  2393618 May 11 13:18 vsphere_client_virgo_4.log.zip
-rw------- 1 vsphere-client users  2420328 May  5 03:24 vsphere_client_virgo_5.log.zip
-rw------- 1 vsphere-client users  2416921 Apr 29 12:26 vsphere_client_virgo_6.log.zip
-rw------- 1 vsphere-client users  2853224 Apr 20 05:56 vsphere_client_virgo_7.log.zip
-rw------- 1 vsphere-client users  2571713 Apr 15 14:16 vsphere_client_virgo_8.log.zip
-rw------- 1 vsphere-client users  2081604 Apr 10 19:19 vsphere_client_virgo_9.log.zip
-rw------- 1 vsphere-client users 19332198 Jun 13 16:03 vspheremessaging.log
-rw------- 1 vsphere-client users  2095797 Jun  3 09:03 vspheremessaging_0.log.zip
-rw------- 1 vsphere-client users  2090504 May 19 00:47 vspheremessaging_1.log.zip
-rw------- 1 vsphere-client users  2127360 May  5 10:53 vspheremessaging_2.log.zip
-rw------- 1 vsphere-client users  2140566 Apr 18 04:18 vspheremessaging_3.log.zip
-rw------- 1 vsphere-client users  2231428 Mar 26 03:08 vspheremessaging_4.log.zip
-rwx------ 1 vsphere-client users  2259049 Feb 19 11:59 vspheremessaging_5.log.zip
-rwx------ 1 vsphere-client users  2193377 Jan 28 22:04 vspheremessaging_6.log.zip
-rwx------ 1 vsphere-client users  2242915 Jan 13 06:08 vspheremessaging_7.log.zip
p111vc01:/var/log/vmware/vsphere-client/logs # less vsphere_client_virgo.log
p111vc01:/var/log/vmware/vsphere-client/logs # cd ..
p111vc01:/var/log/vmware/vsphere-client # cd ..
p111vc01:/var/log/vmware # find -name stats.log
./perfcharts/stats.log
p111vc01:/var/log/vmware # less /perfcharts/stats.log
/perfcharts/stats.log: No such file or directory
p111vc01:/var/log/vmware # less perfcharts/stats.log
p111vc01:/var/log/vmware # date
Mon Jun 13 16:07:09 EDT 2016
p111vc01:/var/log/vmware # less perfcharts/stats.log
Caused by: com.vmware.vim.sso.client.exception.ServerCommunicationException: Error communicating to the remote server https://p222vcpsc.s
t.com/sts/STSService/sterlingbackcheck.com
        at com.vmware.vim.sso.client.impl.SoapBindingImpl.sendMessage(SoapBindingImpl.java:147)
        at com.vmware.vim.sso.client.impl.SoapBindingImpl.sendMessage(SoapBindingImpl.java:81)
        at com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl$RequestResponseProcessor.sendRequest(SecurityTokenServiceImpl.java:769
)
        ... 15 more
Caused by: com.sun.xml.internal.ws.client.ClientTransportException: HTTP transport error: java.net.ConnectException: Connection timed out
        at com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(Unknown Source)
        at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.process(Unknown Source)
        at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.processRequest(Unknown Source)
        at com.sun.xml.internal.ws.transport.DeferredTransportPipe.processRequest(Unknown Source)
        at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Unknown Source)
        at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Unknown Source)
        at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Unknown Source)
        at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Unknown Source)
        at com.sun.xml.internal.ws.client.Stub.process(Unknown Source)
        at com.sun.xml.internal.ws.client.dispatch.DispatchImpl.doInvoke(Unknown Source)
        at com.sun.xml.internal.ws.client.dispatch.DispatchImpl.invoke(Unknown Source)
        at com.vmware.vim.sso.client.impl.SoapBindingImpl.sendMessage(SoapBindingImpl.java:130)
        ... 17 more
Caused by: java.net.ConnectException: Connection timed out
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
        at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
        at sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source)
        at sun.net.NetworkClient.doConnect(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
        at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
perfcharts/stats.log lines 11413-11447/11522 99%