Skip to content

Host capacity calculation: use VM creation time if update time is null.#9662

Merged
JoaoJandre merged 1 commit into
apache:mainfrom
mprokopchuk:update_host_capacity_vm_update_time_fix
Sep 11, 2024
Merged

Host capacity calculation: use VM creation time if update time is null.#9662
JoaoJandre merged 1 commit into
apache:mainfrom
mprokopchuk:update_host_capacity_vm_update_time_fix

Conversation

@mprokopchuk

@mprokopchuk mprokopchuk commented Sep 10, 2024

Copy link
Copy Markdown
Contributor

CapacityManagementImpl.updateCapacityForHost(..) use VM update time in capacity calculation.

VM update time is nullable in DB and can cause NullPointerException if record in vm_instance has defined last_host_id and undefined update_time.

Added logic to check whether update_time is null and use created time instead (which is not null).

Description

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Tested manually.

…n capacity calculation.

VM update time is nullable in DB and can cause NullPointerException if record in vm_instance has defined last_host_id and undefined update_time.
@mprokopchuk

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@mprokopchuk mprokopchuk marked this pull request as ready for review September 10, 2024 03:06
@blueorangutan

Copy link
Copy Markdown

@mprokopchuk a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@mprokopchuk mprokopchuk changed the title Host capacity calculation: use VM creation time if update time is null Host capacity calculation: use VM creation time if update time is null. Sep 10, 2024
@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 11063

@mprokopchuk

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@sureshanaparti

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@sureshanaparti sureshanaparti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@codecov

codecov Bot commented Sep 10, 2024

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 15.77%. Comparing base (501d8c1) to head (90c6eed).
Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
...n/java/com/cloud/capacity/CapacityManagerImpl.java 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9662      +/-   ##
============================================
- Coverage     15.77%   15.77%   -0.01%     
  Complexity    12538    12538              
============================================
  Files          5621     5621              
  Lines        491556   491557       +1     
  Branches      60227    62130    +1903     
============================================
  Hits          77562    77562              
- Misses       405537   405538       +1     
  Partials       8457     8457              
Flag Coverage Δ
uitests 4.05% <ø> (ø)
unittests 16.59% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yadvr yadvr added this to the 4.20.0.0 milestone Sep 10, 2024
@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 11066

@sureshanaparti

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11070

@JoaoJandre JoaoJandre modified the milestones: 4.20.0.0, 4.21.0.0 Sep 10, 2024
@yadvr yadvr modified the milestones: 4.21.0.0, 4.20.1.0 Sep 11, 2024
@yadvr yadvr added the type:bug label Sep 11, 2024
@yadvr

yadvr commented Sep 11, 2024

Copy link
Copy Markdown
Member

@JoaoJandre since this is a bugfix, causing NPE exception can we consider it?

@kiranchavala

Copy link
Copy Markdown
Member

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@kiranchavala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@kiranchavala kiranchavala left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Steps to reproduce the issue

Before fix

  1. Stop a running vm

  2. Manually update the database with the following sql query

mysql>update vm_instance set update_time=NULL where id=20;

  1. Execute the list capacity api call with fetchlatest=true

https://cloudstack.apache.org/api/apidocs-4.19/apis/listCapacity.html

  1. Null pointer exception in the logs
2024-09-11 09:38:53,428 DEBUG [c.c.a.ApiServlet] (qtp660017404-18:ctx-031e6e67) (logid:4fab4036) ===START===  10.0.3.251 -- GET  apiKey=l1_SR4uC25TftqakF--jG9Su3VhMIeR6nvTZairizwO-ZEx_du9PRQsg8efhUP5THn_4vPM1IfzABYMmfkwYig&command=listCapacity&fetchlatest=true&response=json&signature=T8%2FBKjc9uZmnGlCkiAn21RCy6qU%3D


2024-09-11 09:38:53,456 ERROR [c.c.a.AlertManagerImpl] (qtp660017404-18:ctx-031e6e67 ctx-60961f1a) (logid:4fab4036) Caught exception in recalculating capacity
java.lang.NullPointerException
	at com.cloud.capacity.CapacityManagerImpl.updateCapacityForHost(CapacityManagerImpl.java:714)
	at com.cloud.alert.AlertManagerImpl.recalculateCapacity(AlertManagerImpl.java:286)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)

After fix

  1. No Null pointer exceptinon observed

Logs


2024-09-11 10:27:01,862 DEBUG [c.c.a.ApiServlet] (qtp253011924-22:[ctx-8d67dfb3]) (logid:91763e20) ===START===  10.0.3.251 -- GET  apiKey=l1_SR4uC25TftqakF--jG9Su3VhMIeR6nvTZairizwO-ZEx_du9PRQsg8efhUP5THn_4vPM1IfzABYMmfkwYi&command=listCapacity&fetchlatest=true&response=json&signature=p5x0bMfDYCzQyGsSHP4v2PSouGc%3D

2024-09-11 10:27:01,932 INFO  [c.c.a.ApiServlet] (qtp253011924-22:[ctx-8d67dfb3, ctx-ca9e43a9]) (logid:91763e20) (userId=2 accountId=2 sessionId=node01wqd73ie98jsaho1i4b0xibv4) 10.0.3.251 -- GET apiKey=l1_SR4uC25TftqakF--jG9Su3VhMIeR6nvTZairizwO-ZEx_du9PRQsg8efhUP5THn_4vPM1IfzABYMmfkwYi&command=listCapacity&fetchlatest=true&response=json&signature=p5x0bMfDYCzQyGsSHP4v2PSouGc%3D 200 {"listcapacityresponse":{"count":10,"capacity":[{"type":0,"name":"MEMORY","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityallocated":2684354560,"capacityused":2684354560,"capacitytotal":7228305408,"percentused":"37.14"},{"type":1,"name":"CPU","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityallocated":2000,"capacityused":2000,"capacitytotal":12600,"percentused":"15.87"},{"type":2,"name":"STORAGE","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityused":1250223456256,"capacitytotal":2197949513728,"percentused":"56.88"},{"type":3,"name":"STORAGE_ALLOCATED","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityused":24318968016,"capacitytotal":4395899027456,"percentused":"0.55"},{"type":4,"name":"VIRTUAL_NETWORK_PUBLIC_IP","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityused":3,"capacitytotal":20,"percentused":"15"},{"type":5,"name":"PRIVATE_IP","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityused":2,"capacitytotal":20,"percentused":"10"},{"type":6,"name":"SECONDARY_STORAGE","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityused":1250222407680,"capacitytotal":2197949513728,"percentused":"56.88"},{"type":7,"name":"VLAN","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityused":1,"capacitytotal":20,"percentused":"5"},{"type":19,"name":"GPU","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityused":0,"capacitytotal":0,"percentused":"0"},{"type":90,"name":"CPU_CORE","zoneid":"3d129b60-79b8-43aa-bf87-f06d2341b9aa","zonename":"ref-trl-7311-k-Mol8-kiran-chavala","capacityallocated":4,"capacityused":4,"capacitytotal":3,"percentused":"133.33"}]}}

2024-09-11 10:27:01,932 DEBUG [c.c.a.ApiServlet] (qtp253011924-22:[ctx-8d67dfb3, ctx-ca9e43a9]) (logid:91763e20) ===END===  10.0.3.251 -- GET  apiKey=l1_SR4uC25TftqakF--jG9Su3VhMIeR6nvTZairizwO-ZEx_du9PRQsg8efhUP5THn_4vPM1IfzABYMmfkwYi&command=listCapacity&fetchlatest=true&response=json&signature=p5x0bMfDYCzQyGsSHP4v2PSouGc%3D

@JoaoJandre JoaoJandre merged commit 68f459b into apache:main Sep 11, 2024
@blueorangutan

Copy link
Copy Markdown

[SF] Trillian test result (tid-11454)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 86936 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9662-t11454-kvm-ol8.zip
Smoke tests completed. 134 look OK, 3 have errors, 4 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_secure_vm_migration Error 134.14 test_vm_life_cycle.py
test_01_secure_vm_migration Error 134.14 test_vm_life_cycle.py
test_01_VPC_nics_after_destroy Error 787.03 test_vpc_router_nics.py
test_02_VPC_default_routes Error 39.51 test_vpc_router_nics.py
ContextSuite context=TestVPCNics>:teardown Error 81.67 test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpn Failure 982.35 test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn Error 982.36 test_vpc_vpn.py
ContextSuite context=TestRVPCSite2SiteVpn>:teardown Error 1027.31 test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options Failure 865.00 test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options Error 865.01 test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:teardown Error 1163.87 test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup Error 1258.69 test_vpc_vpn.py
test_01_vpc_site2site_vpn Failure 1382.50 test_vpc_vpn.py
test_01_vpc_site2site_vpn Error 1382.52 test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:teardown Error 1421.17 test_vpc_vpn.py
all_test_webhook_delivery Skipped --- test_webhook_delivery.py
all_test_webhook_lifecycle Skipped --- test_webhook_lifecycle.py
all_test_host_maintenance Skipped --- test_host_maintenance.py
all_test_hostha_kvm Skipped --- test_hostha_kvm.py

@yadvr yadvr removed this from the 4.20.1.0 milestone Sep 13, 2024
@yadvr yadvr added this to the 4.20.0.0 milestone Sep 13, 2024
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Sep 23, 2024
…n capacity calculation. (apache#9662)

VM update time is nullable in DB and can cause NullPointerException if record in vm_instance has defined last_host_id and undefined update_time.
@JoaoJandre JoaoJandre mentioned this pull request Oct 8, 2024
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants