⚠ Bofrost.at has degraded performance

In bd962ee, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1354 ms

Resolved: Bofrost.at performance has improved in 6cf936b after 6 minutes.


[Usage]: Maximum Context Length Exceeded Due to Base64-Encoded Image in Prompt

Your current environment
The output of `python collect_env.py`

How would you like to use vllm
Example Code
llm = VLLMOpenAI(
openai_api_key="EMPTY",
openai_api_base=api_base,
model_name="microsoft/Phi-3-vision-128k-instruct",
model_kwargs={"stop": ["."]}
)
image_path = "invoice_data_images/Screenshot 2024-05-02 160946_page_1.png"
with open(image_path, "rb") as image_file:
image_base64 = base64.b64encode(image_file.read()).decode("utf-8")
prompt_1 = "Give me the invoice date from the given image."
messages = [
HumanMessage(
content=[
{"type": "text", "text": prompt_1},
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_base64}"}}
]
)
]
response = llm.invoke(messages)
print(response)
Error Message
{
"name": "BadRequestError",
"message": "Error code: 400 - {'object': 'error', 'message': 'This model's maximum context length is 8192 tokens. However, you requested 254457 tokens (254201 in the messages, 256 in the completion). Please reduce the length of the messages or completion.', 'type': 'BadRequestError', 'param': None, 'code': 400}"
}
Description
I hosted VLLM on an EC2 instance to extract text data from images using the Phi-3 Vision model. The model is hosted with the following command:
python3 -m vllm.entrypoints.openai.api_server --port 8000 --model microsoft/Phi-3-vision-128k-instruct --trust-remote-code --dtype=half --max_model_len=8192
When running the code, I encounter a BadRequestError due to exceeding the maximum context length. The error message indicates that the total number of tokens requested is 254457, which far exceeds the model's limit of 8192 tokens.
The base64-encoded image is being considered part of the prompt, significantly increasing the token count and leading to the context length issue.
Even if the model's context length were 128k tokens, including a base64-encoded image in the prompt would always exceed the model's limit.
Expected Behavior
The model should process the image without considering the base64 string as part of the prompt token count, thereby avoiding the context length issue.

cc @DarkLight1337

I'm a bit confused. How is this issue different from #6301?
To better isolate the issue, please try to use vLLM directly instead of LangChain's integration.

@DarkLight1337
I appreciate the suggestion to use vLLM directly to isolate the issue. However, my goal is to use LangChain agents for this task. How can I integrate LangChain agents with vLLM effectively while managing the context length issue?

I'm not familiar with how the LangChain integration is set up. Perhaps they haven't updated the integration to work with the latest version of vLLM yet? You should open an issue on their repo.

Also I noticed that you have two separate pieces of code here? One is about the LangChain integration but the other one is using vLLM server.

Yes, the command:
python3 -m vllm.entrypoints.openai.api_server --port 8000 --model microsoft/Phi-3-vision-128k-instruct --trust-remote-code --dtype=half --max_model_len=8192
is used to host the microsoft/Phi-3-vision-128k-instruct model.
The hosted API base is then used in the LangChain integration as follows:
llm = VLLMOpenAI(
openai_api_key="EMPTY",
openai_api_base=api_base,
model_name="microsoft/Phi-3-vision-128k-instruct",
model_kwargs={"stop": ["."]}
)

Can you share an image that causes this problem so that I can test it out locally?

I tried some sample images using OpenAI client and failed to trigger this issue.

Perhaps you have to remove model_kwargs={"stop": ["."]}

I attached an image here.
Can you share your code?

I can load this image as normal on my end. Maybe it's a problem with LangChain's integration.
Server:
python3 -m vllm.entrypoints.openai.api_server --port 8001 --model microsoft/Phi-3-vision-128k-instruct --trust-remote-code --dtype=half --max_model_len=8192

Client:
from openai import OpenAI
from vllm.multimodal.utils import ImageFetchAiohttp

openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8001/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

model = client.models.list().data[0].id
print("Model:", model)

image_url = "https://private-user-images.githubusercontent.com/117747869/347734203-bf810dde-3e99-4ee0-b226-455045074e4f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA2ODYwMzUsIm5iZiI6MTcyMDY4NTczNSwicGF0aCI6Ii8xMTc3NDc4NjkvMzQ3NzM0MjAzLWJmODEwZGRlLTNlOTktNGVlMC1iMjI2LTQ1NTA0NTA3NGU0Zi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxMVQwODE1MzVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05MGE1ZmNlYTkxNzdjZWIwYzIzMDMyZDE5ZDhlMWJkZDBiMDg5ZjNkZjhmZGRhYTA2MWYxZjdhY2FmZTk0OWJjJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.CjckU4aV1mV8IksZa8N2C0lnQHthwJMU1alcGFcWbgs"
chat_response = client.chat.completions.create(
    model=model,
    messages=[{
        "role": "user",
        "content": [
            {
                "type": "image_url",
                "image_url": {
                    "url": image_url,
                },
            },
            {"type": "text", "text": "What is in this image?"},
        ],
    }],
)

print("Chat response:", chat_response)

You are using OpenAI, but I want to use LangChain.
from langchain_community.llms import VLLMOpenAI

If this is a problem with LangChain's integration, you'll have to open an issue on LangChain's repo, not here (the integration lives on their repo, after all).

Thanks for your support and time. I will raise an issue on the LangChain repo.

Thanks for your support and time. I will raise an issue on the LangChain repo.


[15.0.x] ISPN-16261 ProtoStream 5.0.7.Final

https://issues.redhat.com/browse/ISPN-16261

@ryanemerson native build failed :cry:

merged! thanks @ryanemerson !


Revisit IsoDate string

There seems to be some problems with the ISO Date string module, not sure what's going on, but none of the validator.js tests were passing

This might be better off as being replace with a dateFromIsoString schemable, and avoid testing for decoding from an IsoString to Date, and back

This has been addressed by #153


Upgrade to mapbox-gl-js

See #51 for original request form @mileszim.

Also, gonna bring this up here, but any interest in updating to the mapbox-gl-js that's the shiny new supported version? It could happen in conjunction with updating the addon base to a more recent ember version.

Keep in mind that mapbox-gl isn't supported in widely used 'enterprise' browsers...


Possibility to speed up further with JAX?

Have you seen Google's JAX project?
https://github.com/google/jax
Do you think it might feasible to speed this up further with that?

Thanks for the suggestion!
Some people suggested me to use JAX, cupy, pytorch etc instead of numpy to speed up calculations with a GPU. I tested using GPU backend pytorch tensors, but the main problem I see is that although it runs faster for single calls, for multiple calls kills the performance due the function call overhead.
I read that JAX can compile several functions using a jit compiler,  so it is likely that some parts can be optimized. But the main problem I'm going to face is  that my source code has a lot recursion and branching.

But the main problem I'm going to face is that my source code has a lot recursion and branching

Yup, that makes sense.

So this means that multiprocessing is also complicated to implement?

So this means that multiprocessing is also complicated to implement?

Multiprocessing works on CPU cores so it doesn't have these GPU limitations. What happens it's that my raytracer already uses multiple cores because lot of Numpy methods uses multithreading so it cannot be made faster using multiprocessing.
About the PyTracer project I already have talked with the author and made some tests. It's also a cool project.
But a pure Python implementation with multiprocessing  isn't faster than a Numpy  implementation on an usual PC. (~4 cores)
The reason is  because Python loops are very slow even running on multiple cores. So throwing the pixel loop to Numpy and make use of their multithreaded compiled methods is the best idea I've seen.


bug - ArkOS 09292024 for PowKitty RGB10x - Drastic hotkey Issue

Describe the Issue (If applicable)
On Powkitty RGB10x ,

which has 1 analog stick.
The hotkey to bring up the Drastic emulator menu is set to the same keycode as the analog stick 'press down' keycode. (stylus click keycode). So this means, in a game which depends on a stylus click, when the user does this, it brings up the Drastic emulator menu....instead of clicking down on the on the graphical item below the stylus.
How can the issue be reproduced? (If applicable)
Open a nintendo ds game, Super Princess Peach (for example), move the stylus over one of the 4 heart icons, and press down on the analog joystick. It will open the Drastic emulator menu.
What device are you using?
Powkitty RGB10x
What version of ArkOS are you on?
Arkos 09292024
Anything else you'd like to include that may help to review this issue or feature request?
I changed my Powkitty RGB10x's
/opt/drastic/config.cfg to this:
frameskip_type = 2
frameskip_value = 1
safe_frameskip = 0
show_frame_counter = 0
screen_orientation = 2
screen_swap = 0
savestate_number = 0
fast_forward = 0
enable_sound = 1
clock_speed = 0
threaded_3d = 1
mirror_touch = 0
compress_savestates = 1
savestate_snapshot = 1
unzip_roms = 0
preload_roms = 0
backup_in_savestates = 1
ignore_gamecard_limit = 0
frame_interval = 0
trim_roms = 0
fix_main_2d_screen = 0
disable_edge_marking = 0
interframe_blend = 0
hires_3d = 0
use_rtc_custom_time = 0
rtc_custom_time = 0
rtc_system_time = 0
slot2_device_type = 0
rumble_frames = 3
firmware.username = Dr DraStic
firmware.language = 1
firmware.favorite_color = 0
firmware.birthday_month = 1
firmware.birthday_day = 1
enable_cheats = 1
controls_a[CONTROL_INDEX_UP] = 1217
controls_a[CONTROL_INDEX_DOWN] = 1153
controls_a[CONTROL_INDEX_LEFT] = 1216
controls_a[CONTROL_INDEX_RIGHT] = 1152
controls_a[CONTROL_INDEX_A] = 1025
controls_a[CONTROL_INDEX_B] = 1024
controls_a[CONTROL_INDEX_X] = 1026
controls_a[CONTROL_INDEX_Y] = 1027
controls_a[CONTROL_INDEX_L] = 1028
controls_a[CONTROL_INDEX_R] = 1029
controls_a[CONTROL_INDEX_START] = 1037
controls_a[CONTROL_INDEX_SELECT] = 1036
controls_a[CONTROL_INDEX_HINGE] = 104
controls_a[CONTROL_INDEX_TOUCH_CURSOR_UP] = 1219
controls_a[CONTROL_INDEX_TOUCH_CURSOR_DOWN] = 1155
controls_a[CONTROL_INDEX_TOUCH_CURSOR_LEFT] = 1218
controls_a[CONTROL_INDEX_TOUCH_CURSOR_RIGHT] = 1154
controls_a[CONTROL_INDEX_TOUCH_CURSOR_PRESS] = 1038
controls_a[CONTROL_INDEX_MENU] = 109
controls_a[CONTROL_INDEX_SAVE_STATE] = 318
controls_a[CONTROL_INDEX_LOAD_STATE] = 320
controls_a[CONTROL_INDEX_FAST_FORWARD] = 8
controls_a[CONTROL_INDEX_SWAP_SCREENS] = 115
controls_a[CONTROL_INDEX_SWAP_ORIENTATION_A] = 97
controls_a[CONTROL_INDEX_SWAP_ORIENTATION_B] = 100
controls_a[CONTROL_INDEX_LOAD_GAME] = 65535
controls_a[CONTROL_INDEX_QUIT] = 65535
controls_a[CONTROL_INDEX_FAKE_MICROPHONE] = 65535
controls_a[CONTROL_INDEX_UI_UP] = 338
controls_a[CONTROL_INDEX_UI_DOWN] = 337
controls_a[CONTROL_INDEX_UI_LEFT] = 336
controls_a[CONTROL_INDEX_UI_RIGHT] = 335
controls_a[CONTROL_INDEX_UI_SELECT] = 13
controls_a[CONTROL_INDEX_UI_BACK] = 8
controls_a[CONTROL_INDEX_UI_EXIT] = 27
controls_a[CONTROL_INDEX_UI_PAGE_UP] = 331
controls_a[CONTROL_INDEX_UI_PAGE_DOWN] = 334
controls_a[CONTROL_INDEX_UI_SWITCH] = 481
controls_b[CONTROL_INDEX_UP] = 1032
controls_b[CONTROL_INDEX_DOWN] = 1033
controls_b[CONTROL_INDEX_LEFT] = 1034
controls_b[CONTROL_INDEX_RIGHT] = 1035
controls_b[CONTROL_INDEX_A] = 65535
controls_b[CONTROL_INDEX_B] = 65535
controls_b[CONTROL_INDEX_X] = 65535
controls_b[CONTROL_INDEX_Y] = 65535
controls_b[CONTROL_INDEX_L] = 65535
controls_b[CONTROL_INDEX_R] = 65535
controls_b[CONTROL_INDEX_START] = 13
controls_b[CONTROL_INDEX_SELECT] = 538
controls_b[CONTROL_INDEX_HINGE] = 65535
controls_b[CONTROL_INDEX_TOUCH_CURSOR_UP] = 65535
controls_b[CONTROL_INDEX_TOUCH_CURSOR_DOWN] = 65535
controls_b[CONTROL_INDEX_TOUCH_CURSOR_LEFT] = 65535
controls_b[CONTROL_INDEX_TOUCH_CURSOR_RIGHT] = 65535
controls_b[CONTROL_INDEX_TOUCH_CURSOR_PRESS] = 1038
controls_b[CONTROL_INDEX_MENU] = 1039
controls_b[CONTROL_INDEX_SAVE_STATE] = 65535
controls_b[CONTROL_INDEX_LOAD_STATE] = 65535
controls_b[CONTROL_INDEX_FAST_FORWARD] = 65535
controls_b[CONTROL_INDEX_SWAP_SCREENS] = 1040
controls_b[CONTROL_INDEX_SWAP_ORIENTATION_A] = 1031
controls_b[CONTROL_INDEX_SWAP_ORIENTATION_B] = 1030
controls_b[CONTROL_INDEX_LOAD_GAME] = 65535
controls_b[CONTROL_INDEX_QUIT] = 65535
controls_b[CONTROL_INDEX_FAKE_MICROPHONE] = 65535
controls_b[CONTROL_INDEX_UI_UP] = 1032
controls_b[CONTROL_INDEX_UI_DOWN] = 1033
controls_b[CONTROL_INDEX_UI_LEFT] = 1034
controls_b[CONTROL_INDEX_UI_RIGHT] = 1035
controls_b[CONTROL_INDEX_UI_SELECT] = 1025
controls_b[CONTROL_INDEX_UI_BACK] = 1026
controls_b[CONTROL_INDEX_UI_EXIT] = 1024
controls_b[CONTROL_INDEX_UI_PAGE_UP] = 1028
controls_b[CONTROL_INDEX_UI_PAGE_DOWN] = 1029
controls_b[CONTROL_INDEX_UI_SWITCH] = 1039

Essentially
I changed analog stick press to 1038
controls_a[CONTROL_INDEX_TOUCH_CURSOR_PRESS] = 1038
controls_b[CONTROL_INDEX_TOUCH_CURSOR_PRESS] = 1038

and I changed
these fields to 1039 ( code for pressing on '+' button on device)
controls_b[CONTROL_INDEX_MENU] = 1039
controls_b[CONTROL_INDEX_UI_SWITCH] = 1039

So now, on the device :

any time I want to bring up the drastic emulator menu, I press on '+'
any time I want to 'click the stylus', I press down on the analog stick

Besides this tiny issue, I find the ArkOS system to be awesome. Thx for creating and maintaining this. :)

Thanks for the settings recommendation.  I knew that was an issue but the same image used for the rgb10x is primarily focused on the rg351mp and I didn't take the time to figure out how to best address this for the rgb10max.  I'll keep this configuration handy for possible future inclusion in a future update.  Enjoy!

Cool. You are welcome :)
On Sat, Nov 30, 2024, 6:46 p.m. christianhaitian @.***>
wrote:

Thanks for the settings recommendation. I knew that was an issue but the
same image used for the rgb10x is primarily focused on the rg351mp and I
didn't take the time to figure out how to best address this for the
rgb10max. I'll keep this configuration handy for possible future inclusion
in a future update. Enjoy!
—
Reply to this email directly, view it on GitHub
https://github.com/christianhaitian/arkos/issues/1229#issuecomment-2509479667,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIM64LLM6QOHFZCYYXQ7F32DJE5JAVCNFSM6AAAAABSY6X4CKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGQ3TSNRWG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

Cool.
Maybe put it in the readme or wiki
under RGB10x category or something?
Just a thought. :)
On Sat, Nov 30, 2024, 6:46 p.m. christianhaitian @.***>
wrote:

Thanks for the settings recommendation. I knew that was an issue but the
same image used for the rgb10x is primarily focused on the rg351mp and I
didn't take the time to figure out how to best address this for the
rgb10max. I'll keep this configuration handy for possible future inclusion
in a future update. Enjoy!
—
Reply to this email directly, view it on GitHub
https://github.com/christianhaitian/arkos/issues/1229#issuecomment-2509479667,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIM64LLM6QOHFZCYYXQ7F32DJE5JAVCNFSM6AAAAABSY6X4CKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGQ3TSNRWG4
.
You are receiving this because you authored the thread.Message ID:
@.***>


Enhanced documentation with Bioscrape v1.2.2

Dev to master

I rebased the changes from master on dev so that the dev and master are not diverged anymore. It seems that it worked...In the repository Insights -> Network, I see that dev has all the commits now replayed on top. So, hopefully, after this merge to master dev and master can be together and we don't see 3 year old commits every time we merge a PR.


All primitive types are marked as "required" in Schema regardless of JsonProperty(required=false} is set

Given the following simple code:
public class JsonSchemaCreator {

  public static void main(String[] args) throws JsonProcessingException {
    ObjectMapper objectMapper = new ObjectMapper();
    JsonSchemaGenerator jsonSchemaGenerator = new JsonSchemaGenerator(objectMapper);
    JsonNode jsonSchema = jsonSchemaGenerator.generateJsonSchema(Foo.class);
    String jsonSchemaAsString = objectMapper.writeValueAsString(jsonSchema);
    System.out.println(jsonSchemaAsString);
  }

  public static class Foo {
    public Foo() {}

    @JsonProperty()
    private String string1;
    @JsonProperty(required = false)
    private int int1 = 0;
    @JsonProperty()
    private boolean bool1;
  }

The following output is created:
{"$schema":"http://json-schema.org/draft-04/schema#","title":"Foo","type":"object","additionalProperties":false,"properties":{"string1":{"type":"string"},"int1":{"type":"integer"},"bool1":{"type":"boolean"}},"required":["int1","bool1"]}

As you can see, both of the primitive fields "int1" and "bool1" are marked as required.
I found the following in the Scala source, which makes it look like this behavior is by design?  I would expect the JsonProperty(required=true/false) to be honored.  Any feedback or workaround would be appreciated.
// Check if we should set this property as required. Primitive types MUST have a value, as does anything
// with a @JsonProperty that has "required" set to true. Lastly, various javax.validation annotations also
// make this required.

val requiredProperty:Boolean = if (propertyType.getRawClass.isPrimitive || jsonPropertyRequired || validationAnnotationRequired(prop)) {
       true
  } else {
       false
  }

Any feedback or workaround would be appreciated.
Thanks

Hi,
It is like this "by design".
A primitive cannot NOT hold a value.
In your code, the property ** int1** will end up with the value 0, if your exclude it from the json.
If you would like to allow NOT having a value, you should use Integer (with a capital I) instead. Then it will end up with null, if the value is missing. Or you could use Optional (java) or Option (Scala).
I hope this helps.

Ah I see,  it is a marshalling concern, which  makes sense.  Thanks for the quick response.

@mbknor You're mistaken. "required" is not about what the Java code is doing, it's about what we expect from input and output documents. Jackson allows various patterns using setters, default values, tracking whether a field has been set, etc. (In particular I'm using @Jacksonized @Builder which exposes the setter pattern and default values.)
A setter which takes a primitive value is not required! I'm annoyed by this as well.
I think the real crux of the issue is serialization-vs-deserialization. A primitive field is "required" on serialization since it's always set. However, I'm using this library only to generate schemas for deserialization. This issue crops up in multiple other places, like how this library generates schema based on the class's serializable properties instead of the deserializable properties (ie, those in the Creator method).

I don't want the schema generator to guess what is or isn't required. It should just follow JsonProperty(required).


Black background square on hover fixed button

Look at the right bottom button, there is a black background square on hover.
It's new, previous version was good on this point.
Tested on Ubuntu Mate 14.04.02/Firefox 36.0.4 with last downloaded archive of Materialize from Github.

Fixed in 4d740bcb928819686daa78f20b592335523b1c29


Support at least back to R 3.6

Make changes to support at least to R version 3.6

This is going to be tricky.  Really hard to debug.  Lot of issues with 3.6 creating factors when it is not supposed to, or subsetting in weird ways.

Several issues had to be fixed.  Lots of test cases fixed.  data.frame() function needed stringsAsFactors = FALSE. Also there was a bug in the fmtr fdata() function where it was returning all factors on R 3.6.  Took a while to fix everything.


Add clamped parameter to internal devices

Rather than:
from gpiozero import LEDBarGraph, CPUTemperature
from gpiozero.tools import clamped

leds = LEDBarGraph(2, 3, 4, 5, 6 pwm=True)
cpu = CPUTemperature()

leds.source = clamped(cpu.values, 0, 1)

would it be nice to be able to:
from gpiozero import LEDBarGraph, CPUTemperature

leds = LEDBarGraph(2, 3, 4, 5, 6 pwm=True)
cpu = CPUTemperature(clamped=True)

leds.source = cpu.values

?

Personally, I don't think this suggestion makes sense... why assume that clamped always means 0->1? Why only support clamped and not also booleanized? etc. etc.
BTW the code here doesn't use clamped - does it not work properly?

It does work until the value goes outside the range, and you get an OutputDeviceBadValue exception. I think it kills the source thread at that point. But I don't think it's worth doing anything about it - as you say, why not for other similar cases? I'll drop it.


Python Wrapper: Bug bash fixes

Change Description
Background
test_object.py

verify_reader can be static - DONE
conf.py
copyright shadows built-in name - DONE
test_object.py
range - build-in name - NOT DONE this is a mock of the lakefs-sdk function signature
repository.py
repo used from outer scope - DONE
unknown property - dynamic attribute will be marked as trying to set as error - DONE
object.py
close - stat object is not the from the API, so there is no dict. - DONE - modified return values

Thank you!


Kubernetes nodepool "Parameter Reference" documentation bug - Parameter types are incorrect

Description
On running the k8s nodepool task ionoscloudsdk.ionoscloud.k8s_nodepool with state="update", we get "500 - Internal Server Error". Requesting help to diagnose the error or to realise any issues with our task inputs. NB// this is possibly not a Bug, maybe only in so much as the behaviour of the module is possibly different from the docs.
Expected behavior
The following sequence:
(1) run k8s nodepool task ionoscloudsdk.ionoscloud.k8s_nodepool with state="create" and node_count="1". Expectation: a nodepool is created with node_count="1". Result: the nodepool is created correctly.
(2) run k8s nodepool task ionoscloudsdk.ionoscloud.k8s_nodepool with state="update" and node_count="2". Expectation: the above created nodepool is updated so that node_count="2". Result: 500 - Internal Server Error.
Environment
Ansible version:
ansible==5.8.0
ansible-core==2.12.6
ionoscloud==6.0.4

Module version:
ionoscloudsdk.ionoscloud 6.2.0

OS:
Ubuntu (Focal)

How to Reproduce
Steps to reproduce the behaviour:

Following is the Invocation log for k8s nodepool task ionoscloudsdk.ionoscloud.k8s_nodepool with state="create" and node_count="1":

2022-06-08T14:37:45.6618681Z     "invocation": {
2022-06-08T14:37:45.6619014Z         "module_args": {
2022-06-08T14:37:45.6619371Z             "annotations": null,
2022-06-08T14:37:45.6619666Z             "api_url": null,
2022-06-08T14:37:45.6619948Z             "auto_scaling": {
2022-06-08T14:37:45.6620235Z                 "max_node_count": "4",
2022-06-08T14:37:45.6620610Z                 "min_node_count": "1"
2022-06-08T14:37:45.6620884Z             },
2022-06-08T14:37:45.6621254Z             "availability_zone": "AUTO",
2022-06-08T14:37:45.6621643Z             "cores_count": "1",
2022-06-08T14:37:45.6621957Z             "cpu_family": "INTEL_SKYLAKE",
2022-06-08T14:37:45.6622628Z             "datacenter_id": "68e1a018-eaf5-40a4-b416-60d3ae6732ae",
2022-06-08T14:37:45.6623224Z             "k8s_cluster_id": "4dd2418f-fd65-4623-92f4-f89f135522c1",
2022-06-08T14:37:45.6623587Z             "k8s_version": "1.23.6",
2022-06-08T14:37:45.6623869Z             "labels": {
2022-06-08T14:37:45.6624155Z                 "status": "the_jungle_vip"
2022-06-08T14:37:45.6624434Z             },
2022-06-08T14:37:45.6624695Z             "lan_ids": null,
2022-06-08T14:37:45.6624987Z             "maintenance_window": {
2022-06-08T14:37:45.6626463Z                 "day_of_the_week": "Sunday",
2022-06-08T14:37:45.6626797Z                 "time": "03:45:00"
2022-06-08T14:37:45.6627064Z             },
2022-06-08T14:37:45.6627319Z             "node_count": 1,
2022-06-08T14:37:45.6627723Z             "nodepool_id": null,
2022-06-08T14:37:45.6630684Z             "nodepool_name": "king_of_the_swingers",
2022-06-08T14:37:45.6631884Z             "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
2022-06-08T14:37:45.6632229Z             "public_ips": null,
2022-06-08T14:37:45.6632517Z             "ram_size": "2048",
2022-06-08T14:37:45.6632805Z             "state": "present",
2022-06-08T14:37:45.6637384Z             "storage_size": "10",
2022-06-08T14:37:45.6637777Z             "storage_type": "SSD",
2022-06-08T14:37:45.6638136Z             "token": null,
2022-06-08T14:37:45.6639056Z             "username": "***",
2022-06-08T14:37:45.6639350Z             "wait": true,
2022-06-08T14:37:45.6639630Z             "wait_timeout": 600
2022-06-08T14:37:45.6639886Z         }
2022-06-08T14:37:45.6640117Z     }

For the above invocation with state="create", the task performs as expected. Here is the corresponding log:
2022-06-08T14:37:45.6640371Z     "nodepool": {
2022-06-08T14:37:45.6641081Z         "href": "https://api.ionos.com/cloudapi/v6/k8s/4dd2418f-fd65-4623-92f4-f89f135522c1/nodepools/e93e937c-c089-4046-8bee-87eae77be578",
2022-06-08T14:37:45.6641916Z         "id": "e93e937c-c089-4046-8bee-87eae77be578",
2022-06-08T14:37:45.6654525Z         "metadata": {
2022-06-08T14:37:45.6655237Z             "created_by": "***",
2022-06-08T14:37:45.6655815Z             "created_by_user_id": "fe32a0f2-b40f-4332-846c-93dec3750045",
2022-06-08T14:37:45.6656612Z             "created_date": "2022-06-08T14:28:42+00:00",
2022-06-08T14:37:45.6657024Z             "etag": "3ad7bb1fcf47d74c51a2781d362b6e11",
2022-06-08T14:37:45.6657564Z             "last_modified_by": "***",
2022-06-08T14:37:45.6658273Z             "last_modified_by_user_id": "fe32a0f2-b40f-4332-846c-93dec3750045",
2022-06-08T14:37:45.6658872Z             "last_modified_date": "2022-06-08T14:28:42+00:00",
2022-06-08T14:37:45.6659236Z             "state": "DEPLOYING"
2022-06-08T14:37:45.6659500Z         },
2022-06-08T14:37:45.6659766Z         "properties": {
2022-06-08T14:37:45.6660055Z             "annotations": null,
2022-06-08T14:37:45.6660340Z             "auto_scaling": {
2022-06-08T14:37:45.6660645Z                 "max_node_count": 4,
2022-06-08T14:37:45.6660950Z                 "min_node_count": 1
2022-06-08T14:37:45.6661209Z             },
2022-06-08T14:37:45.6661604Z             "availability_zone": "AUTO",
2022-06-08T14:37:45.6661937Z             "available_upgrade_versions": null,
2022-06-08T14:37:45.6662248Z             "cores_count": 1,
2022-06-08T14:37:45.6662554Z             "cpu_family": "INTEL_SKYLAKE",
2022-06-08T14:37:45.6663101Z             "datacenter_id": "68e1a018-eaf5-40a4-b416-60d3ae6732ae",
2022-06-08T14:37:45.6663471Z             "k8s_version": "1.23.6",
2022-06-08T14:37:45.6663754Z             "labels": {
2022-06-08T14:37:45.6664049Z                 "status": "the_jungle_vip"
2022-06-08T14:37:45.6664315Z             },
2022-06-08T14:37:45.6664560Z             "lans": [],
2022-06-08T14:37:45.6664839Z             "maintenance_window": {
2022-06-08T14:37:45.6665154Z                 "day_of_the_week": "Sunday",
2022-06-08T14:37:45.6665460Z                 "time": "03:45:00Z"
2022-06-08T14:37:45.6665728Z             },
2022-06-08T14:37:45.6666015Z             "name": "king_of_the_swingers",
2022-06-08T14:37:45.6666316Z             "node_count": 1,
2022-06-08T14:37:45.6666597Z             "public_ips": null,
2022-06-08T14:37:45.6666873Z             "ram_size": 2048,
2022-06-08T14:37:45.6667162Z             "storage_size": 10,
2022-06-08T14:37:45.6667450Z             "storage_type": "SSD"
2022-06-08T14:37:45.6667705Z         },
2022-06-08T14:37:45.6667964Z         "type": "nodepool"
2022-06-08T14:37:45.6668214Z     },
2022-06-08T14:37:45.6668579Z     "results_file": "/home/***/.ansible_async/328711158290.399884",
2022-06-08T14:37:45.6668914Z     "started": 1,
2022-06-08T14:37:45.6669170Z     "stderr": "",
2022-06-08T14:37:45.6669439Z     "stderr_lines": [],
2022-06-08T14:37:45.6669698Z     "stdout": "",
2022-06-08T14:37:45.6669961Z     "stdout_lines": []
2022-06-08T14:37:45.6670206Z }

Following is the Invocation log for k8s nodepool task ionoscloudsdk.ionoscloud.k8s_nodepool with state="update" and node_count="2". NB// The nodepool_id parameter matches the value output above.

2022-06-08T15:04:37.1363032Z     "invocation": {
2022-06-08T15:04:37.1363415Z         "module_args": {
2022-06-08T15:04:37.1363809Z             "annotations": null,
2022-06-08T15:04:37.1364209Z             "api_url": null,
2022-06-08T15:04:37.1364604Z             "auto_scaling": {
2022-06-08T15:04:37.1365011Z                 "max_node_count": "4",
2022-06-08T15:04:37.1365422Z                 "min_node_count": "1"
2022-06-08T15:04:37.1365796Z             },
2022-06-08T15:04:37.1366191Z             "availability_zone": null,
2022-06-08T15:04:37.1366614Z             "cores_count": null,
2022-06-08T15:04:37.1367023Z             "cpu_family": null,
2022-06-08T15:04:37.1367432Z             "datacenter_id": null,
2022-06-08T15:04:37.1368135Z             "k8s_cluster_id": "4dd2418f-fd65-4623-92f4-f89f135522c1",
2022-06-08T15:04:37.1368639Z             "k8s_version": "1.23.6",
2022-06-08T15:04:37.1369039Z             "labels": {
2022-06-08T15:04:37.1369443Z                 "status": "the_jungle_vip"
2022-06-08T15:04:37.1369834Z             },
2022-06-08T15:04:37.1370202Z             "lan_ids": null,
2022-06-08T15:04:37.1370609Z             "maintenance_window": {
2022-06-08T15:04:37.1371037Z                 "day_of_the_week": "Sunday",
2022-06-08T15:04:37.1371476Z                 "time": "03:45:00"
2022-06-08T15:04:37.1371859Z             },
2022-06-08T15:04:37.1372227Z             "node_count": 2,
2022-06-08T15:04:37.1372877Z             "nodepool_id": "e93e937c-c089-4046-8bee-87eae77be578",
2022-06-08T15:04:37.1373372Z             "nodepool_name": null,
2022-06-08T15:04:37.1373840Z             "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
2022-06-08T15:04:37.1374301Z             "public_ips": null,
2022-06-08T15:04:37.1374708Z             "ram_size": null,
2022-06-08T15:04:37.1375106Z             "state": "update",
2022-06-08T15:04:37.1375513Z             "storage_size": null,
2022-06-08T15:04:37.1375918Z             "storage_type": null,
2022-06-08T15:04:37.1376316Z             "token": null,
2022-06-08T15:04:37.1376908Z             "username": "***",
2022-06-08T15:04:37.1377317Z             "wait": true,
2022-06-08T15:04:37.1377711Z             "wait_timeout": 600
2022-06-08T15:04:37.1378076Z         }
2022-06-08T15:04:37.1378413Z     }

For the above invocation with state="update" the task fails. Here is the corresponding log:
2022-06-08T15:04:37.1380608Z     "msg": "failed to update the nodepool: (500)\nReason: Internal Server Error\nHTTP response headers: HTTPHeaderDict({'Server': 'nginx', 'Date': 'Wed, 08 Jun 2022 15:04:20 GMT', 'Content-Type': 'application/json', 'Content-Length': '160', 'Connection': 'close', 'X-Frame-Options': 'SAMEORIGIN', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Headers': '*', 'Access-Control-Expose-Headers': '*', 'Access-Control-Allow-Methods': '*', 'Access-Control-Allow-Credentials': 'true'})\nHTTP response body: {\n  \"httpStatus\" : 500,\n  \"messages\" : [ {\n    \"errorCode\" : \"301\",\n    \"message\" : \"Oops! Something went very wrong. Please contact the administrator\"\n  } ]\n}\n\n"

Additional Notes
Observe which parameters are set to null in the invocation logs. We have tried to ensure null is passed for at least any parameters that are not explicitly handled by the underlying python function for the "create" and "update" cases. In the "update" case, we are looking at parameters handled by function update_k8s_cluster_nodepool() in k8s_nodepool.py. For example, cores_count is set to null. We believe this approach is also consistent with the Examples and Parameter Reference in the docs.
References
#54

Hello @detmoo, I can't seem to reproduce the error. Is this still happening? It seems like the call is passed to the API correctly by the ansible module, and fails. Can you try running the playbook again?

Hello @avirtopeanu-ionos I am still experiencing the same issue. I agree the call is passed OK to the API but it is not clear what is causing the API to issue the 500 error and therefore, based on the evidence I have, it is not possible to rule out a flaw in the module on transposing parameters from the Ansible playbook to the underlying API call.
Since you have been unable to recreate the error, so please can you share the invocation logs generated on successfully processing your 'update' playbook so that I can reconcile against what I have provided above?
For your information, I have attempted to validate the parameter transpose by tracing the lineage from the parameter interface implemented by the Ansible module through to the parameter interface implemented by the underlying Python SDK (i.e. KubernetesNodepoolPropertiesForPut but I have not found any issues. Should the problem be connected to parameters, to me it might be reasonable to expect successfully parsing the parameter interface in Ansible to cause deterministically successful parsing of the parameter interface for the underlying API call.

Running the following playbook is fine for me
- name: Detmoo issue97
  hosts: localhost
  connection: local
  gather_facts: false

  vars_files:
    - ../vars.yml

  tasks:
    - name: Create datacenter
      datacenter:
        name: "{{ datacenter }}"
        description: "{{ description }}"
        location: "{{ location }}"
      register: datacenter_response

    - name: Create k8s cluster
      k8s_cluster:
        cluster_name: "{{ cluster_name }}"
        maintenance_window:
          day_of_the_week: "Wednesday"
          time: "12:02:00"
        wait: true
      register: cluster

    - name: Create k8s cluster nodepool
      k8s_nodepool:
        nodepool_name: "{{ nodepool_name }}"
        k8s_cluster_id: "{{ cluster.cluster.id }}"
        datacenter_id: "{{ datacenter_response.datacenter.id }}"
        node_count: 1
        cpu_family: "INTEL_SKYLAKE"
        cores_count: "1"
        ram_size: "2048"
        storage_type: null
        storage_size: null
        availability_zone: "AUTO"
        storage_type: "SSD"
        storage_size: "10"
        public_ips: null
        labels:
          status: "the_jungle_vip"
        annotations: null
        auto_scaling:
          max_node_count: 4
          min_node_count: 1
        wait: true
        state: present
      register: result

    - name: Debug - Show nodepool
      debug:
         msg: "{{ result.nodepool }}"

    - name: Update k8s cluster nodepool
      k8s_nodepool:
        k8s_cluster_id: "{{ cluster.cluster.id }}"
        datacenter_id: "{{ datacenter_response.datacenter.id }}"
        nodepool_id: "{{ result.nodepool.id }}"
        node_count: 2
        cpu_family: null
        cores_count: null
        ram_size: null
        # cpu_family: "INTEL_SKYLAKE"
        # cores_count: "1"
        # ram_size: "2048"
        availability_zone: null
        storage_type: null
        storage_size: null
        # availability_zone: "AUTO"
        # storage_type: "HDD"
        # storage_size: "100"
        public_ips: null
        # public_ips: "{{ ipblock_response_update.ipblock.properties.ips }}"
        labels:
          status: "the_jungle_vip"
        annotations: null
        auto_scaling:
          max_node_count: 4
          min_node_count: 1
        wait: true
        state: update

In the meantime I've created a ticket for our API team to investigate the underlying API calls, and I'll return with a comment here once I have more updates

With some modifications* it has been possible to run the Playbook that you provided above. Using this outcome I anticipate to deduce a malformed parameter in my original playbook invocation. Nevertheless, there are still some issues with the results of the above playbook:-
(*For your information - Q. What modifications were necessary to complete the playbook run? A. (i) running on an Azure hosted agent it is necessary to apply async and poll parameters to the playbook - otherwise the Ansible runtime hangs on the first task, even though it will successfully create but not update the nodepool; (ii) we adopted the fully-qualified name for referencing the Ionoscloud Tasks.)
Using the necessary modifications for my environment I can now see the playbook running to completion - but the nodepool still does not conform to the updated/declared specification. Following is (a) from the Ansible logs, observe we are declaring the nodepool shall be updated to have 2 nodes (updated from 1 node) (b) response from ionoscloud.KubernetesApi().k8s_nodepools_get(), observe the resource is in a terminal state ("ACTIVE") but with 'node_count': 1 (we expect 2, given the completion of the playbook). Please can you examine results from your running the playbook and advise if the outcome I am seeing is recreated in your environment?
(a) Ansible logs - declares update of node count so that 'node_count': 2
2022-07-11T15:56:42.3020337Z changed: [localhost] => {
2022-07-11T15:56:42.3020695Z     "action": "update",
2022-07-11T15:56:42.3025168Z     "ansible_job_id": "865076800348.17353",
2022-07-11T15:56:42.3025541Z     "changed": true,
2022-07-11T15:56:42.3025834Z     "finished": 1,
2022-07-11T15:56:42.3026104Z     "invocation": {
2022-07-11T15:56:42.3026396Z         "module_args": {
2022-07-11T15:56:42.3026706Z             "annotations": null,
2022-07-11T15:56:42.3027015Z             "api_url": null,
2022-07-11T15:56:42.3027318Z             "auto_scaling": {
2022-07-11T15:56:42.3027836Z                 "max_node_count": 4,
2022-07-11T15:56:42.3028159Z                 "min_node_count": 1
2022-07-11T15:56:42.3028440Z             },
2022-07-11T15:56:42.3030331Z             "availability_zone": null,
2022-07-11T15:56:42.3030777Z             "cores_count": null,
2022-07-11T15:56:42.3031170Z             "cpu_family": null,
2022-07-11T15:56:42.3036166Z             "datacenter_id": "54f7da8c-1ed6-4651-bef7-36ee648edc5d",
2022-07-11T15:56:42.3039320Z             "k8s_cluster_id": "83493f82-a318-4de1-972e-99495740ea88",
2022-07-11T15:56:42.3041651Z             "k8s_version": null,
2022-07-11T15:56:42.3042184Z             "labels": {
2022-07-11T15:56:42.3044164Z                 "status": "the_jungle_vip"
2022-07-11T15:56:42.3044663Z             },
2022-07-11T15:56:42.3044962Z             "lan_ids": null,
2022-07-11T15:56:42.3045484Z             "maintenance_window": null,
2022-07-11T15:56:42.3045996Z             "node_count": 2,
2022-07-11T15:56:42.3047010Z             "nodepool_id": "538d5abc-82e1-4cb3-b610-b56d8ce3c6a8",
2022-07-11T15:56:42.3047701Z             "nodepool_name": null,
2022-07-11T15:56:42.3048804Z             "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
2022-07-11T15:56:42.3049665Z             "public_ips": null,
2022-07-11T15:56:42.3050528Z             "ram_size": null,
2022-07-11T15:56:42.3050987Z             "state": "update",
2022-07-11T15:56:42.3055208Z             "storage_size": null,
2022-07-11T15:56:42.3055572Z             "storage_type": null,
2022-07-11T15:56:42.3056038Z             "token": null,
2022-07-11T15:56:42.3057481Z             "username": "***",
2022-07-11T15:56:42.3057857Z             "wait": true,
2022-07-11T15:56:42.3058201Z             "wait_timeout": 600
2022-07-11T15:56:42.3058530Z         }
2022-07-11T15:56:42.3058816Z     },
2022-07-11T15:56:42.3059127Z     "nodepool": {
2022-07-11T15:56:42.3060544Z         "href": "https://api.ionos.com/cloudapi/v6/k8s/83493f82-a318-4de1-972e-99495740ea88/nodepools/538d5abc-82e1-4cb3-b610-b56d8ce3c6a8",
2022-07-11T15:56:42.3061985Z         "id": "538d5abc-82e1-4cb3-b610-b56d8ce3c6a8",
2022-07-11T15:56:42.3064090Z         "metadata": {
2022-07-11T15:56:42.3066756Z             "created_by": "***",
2022-07-11T15:56:42.3069551Z             "created_by_user_id": "fe32a0f2-b40f-4332-846c-93dec3750045",
2022-07-11T15:56:42.3072144Z             "created_date": "2022-07-11T15:41:17+00:00",
2022-07-11T15:56:42.3074347Z             "etag": "b787c06ddb5dcaddcf2e9463faa31c9e",
2022-07-11T15:56:42.3075860Z             "last_modified_by": "***",
2022-07-11T15:56:42.3078097Z             "last_modified_by_user_id": "fe32a0f2-b40f-4332-846c-93dec3750045",
2022-07-11T15:56:42.3080979Z             "last_modified_date": "2022-07-11T15:51:02+00:00",
2022-07-11T15:56:42.3081408Z             "state": "UPDATING"
2022-07-11T15:56:42.3082139Z         },
2022-07-11T15:56:42.3082684Z         "properties": {
2022-07-11T15:56:42.3083018Z             "annotations": {},
2022-07-11T15:56:42.3083350Z             "auto_scaling": {
2022-07-11T15:56:42.3084043Z                 "max_node_count": 4,
2022-07-11T15:56:42.3084577Z                 "min_node_count": 1
2022-07-11T15:56:42.3085248Z             },
2022-07-11T15:56:42.3085863Z             "availability_zone": "AUTO",
2022-07-11T15:56:42.3086301Z             "available_upgrade_versions": [
2022-07-11T15:56:42.3086690Z                 "1.22.9",
2022-07-11T15:56:42.3087035Z                 "1.22.11"
2022-07-11T15:56:42.3088748Z             ],
2022-07-11T15:56:42.3089217Z             "cores_count": 1,
2022-07-11T15:56:42.3091035Z             "cpu_family": "INTEL_SKYLAKE",
2022-07-11T15:56:42.3091873Z             "datacenter_id": "54f7da8c-1ed6-4651-bef7-36ee648edc5d",
2022-07-11T15:56:42.3094220Z             "k8s_version": "1.22.8",
2022-07-11T15:56:42.3094561Z             "labels": {
2022-07-11T15:56:42.3094893Z                 "status": "the_jungle_vip"
2022-07-11T15:56:42.3095459Z             },
2022-07-11T15:56:42.3095751Z             "lans": [],
2022-07-11T15:56:42.3096607Z             "maintenance_window": {
2022-07-11T15:56:42.3096966Z                 "day_of_the_week": "Sunday",
2022-07-11T15:56:42.3097323Z                 "time": "13:02:17Z"
2022-07-11T15:56:42.3097612Z             },
2022-07-11T15:56:42.3098352Z             "name": "debugNodepool",
2022-07-11T15:56:42.3098713Z             "node_count": 2,
2022-07-11T15:56:42.3099149Z             "public_ips": null,
2022-07-11T15:56:42.3099582Z             "ram_size": 2048,
2022-07-11T15:56:42.3101547Z             "storage_size": 10,
2022-07-11T15:56:42.3102117Z             "storage_type": "SSD"
2022-07-11T15:56:42.3103906Z         },
2022-07-11T15:56:42.3104233Z         "type": "nodepool"
2022-07-11T15:56:42.3105517Z     },
2022-07-11T15:56:42.3105980Z     "results_file": "/home/***/.ansible_async/865076800348.17353",
2022-07-11T15:56:42.3106641Z     "started": 1,
2022-07-11T15:56:42.3106964Z     "stderr": "",
2022-07-11T15:56:42.3107287Z     "stderr_lines": [],
2022-07-11T15:56:42.3108780Z     "stdout": "",
2022-07-11T15:56:42.3110644Z     "stdout_lines": []
2022-07-11T15:56:42.3113624Z }

(b) ionoscloud.KubernetesApi().k8s_nodepools_get() shows 'node_count': 1 (not 2)
{'href': 'https://api.ionos.com/cloudapi/v6/k8s/83493f82-a318-4de1-972e-99495740ea88/nodepools',
 'id': '83493f82-a318-4de1-972e-99495740ea88/nodepools',
 'items': [{'href': 'https://api.ionos.com/cloudapi/v6/k8s/83493f82-a318-4de1-972e-99495740ea88/nodepools/538d5abc-82e1-4cb3-b610-b56d8ce3c6a8',
            'id': '538d5abc-82e1-4cb3-b610-b56d8ce3c6a8',
            'metadata': {'created_by'<EMAIL_ADDRESS>                         'created_by_user_id': 'fe32a0f2-b40f-4332-846c-93dec3750045',
                         'created_date': datetime.datetime(2022, 7, 11, 15, 41, 17, tzinfo=tzutc()),
                         'etag': '3406c3baa51c6333596989b239ef19df',
                         'last_modified_by'<EMAIL_ADDRESS>                         'last_modified_by_user_id': 'fe32a0f2-b40f-4332-846c-93dec3750045',
                         'last_modified_date': datetime.datetime(2022, 7, 11, 15, 51, 2, tzinfo=tzutc()),
                         'state': 'ACTIVE'},
            'properties': {'annotations': {},
                           'auto_scaling': {'max_node_count': 4,
                                            'min_node_count': 1},
                           'availability_zone': 'AUTO',
                           'available_upgrade_versions': ['1.22.9', '1.22.11'],
                           'cores_count': 1,
                           'cpu_family': 'INTEL_SKYLAKE',
                           'datacenter_id': '54f7da8c-1ed6-4651-bef7-36ee648edc5d',
                           'k8s_version': '1.22.8',
                           'labels': {'status': 'the_jungle_vip'},
                           'lans': [],
                           'maintenance_window': {'day_of_the_week': 'Sunday',
                                                  'time': '13:02:17Z'},
                           'name': 'debugNodepool',
                           'node_count': 1,
                           'public_ips': None,
                           'ram_size': 2048,
                           'storage_size': 10,
                           'storage_type': 'SSD'},
            'type': 'nodepool'}],
 'type': 'collection'}

@avirtopeanu-ionos FYI - I am still working on the evidence provided in the previous post. Possibly we can now close this issue but a new one could be opened concerning what is described above. I will update or close issues shortly. Appreciate any more feedback from you in the mean time.

What's happening is that the nodepool is updated, node_count is set to two, until the autoscaler kicks in (see provided arguments: auto_scaling': {'max_node_count': 4, min_node_count': 1}, and then the autoscaler scales it back down to 1, so this is pretty much intended behaivour.
you can either remove the auto_scaling parameter (if you don't intend this to happen) or to raise its min_node_count to 2

Hi, thank you for the explanation regarding the auto-scaling.
Regarding the original issue. We have indeed deduced there was a malformed parameter in the update task. Namely, we were passing strings for the values of min_node_count and max_node_count. Once we change these fields to integers, the update task runs to completion. Nevertheless, the expectation of integers in the update case is obscure since it looks like the create task works fine with strings for these fields.
As a general point, maybe there is some inconsistency with the use of integers (node_count, min_node_count, max_node_count) and strings (ram_size, storage_size, cores_count) that could be clearer. FYI - as a workaround solution to #54, I have created a generic jinja2 template for all operations on nodepools (i.e.  for state in {"create" | "update" | "delete"}) with some of the parameter handling performed via jinja2 filters. And in this scenario it is arguably preferable to accept strings where possible, to avoid this issue with jinja2.
I will appreciate any updates on these points regarding integers/strings but unless you have further feedback I shall close this issue. Thank you very much.

FYI - check this page from the ansible module docsand observe "Example Syntax". In Create k8s cluster nodepool the integer-like data are represented as a mixture of int and str including node_count (int). In Update k8s cluster nodepool we have node_count (str). Then obsever "Parameter Reference" and you have node_count (int).

@avirtopeanu-ionos further to the comments above regarding need to wrangle the autoscaling parameters min_node_count and max_node_count as integers (cf. it is necessary to pass integers to the update task even though the create task accepts strings) - in fact it seems to be not possible to wrangle the required integers in Ansible jinja2 templates. Therefore, I kindly and respectfully request that you make allowance for this when you refactor the module in respect of the issues I raise here and here.
For me, you can close this issue once you have assimilated. Let me know if there any questions.

Hi, I will re-open this issue, and change its title accordingly. I agree that there is a bug within the documentation regarding the parameter types.
Regarding the declarative tasks, the jinja2 templating is a smart workaround, but a long-term goal is that Ansible should guarantee the properties of resources (which it currently doesn't for create - and #54 is a testament to this)
We really appreciate the feedback, and please keep in mind that your issues are very helpful to this repo's development.

This should be solved in 7.4.1, please let us know if there are other issues.


Revert "fix(SILVA-558): opening details prompt modal"

Reverts bcgov/nr-silva#471
Please see https://github.com/bcgov/nr-silva/pull/475

Thanks for the PR!
Any successful deployments (not always required) will be available below.
Backend: https://nr-silva-476-backend.apps.silver.devops.gov.bc.ca/actuator/health
Frontend: https://nr-silva-26-frontend.apps.silver.devops.gov.bc.ca
Once merged, code will be promoted and handed off to following workflow run.
Main Merge Workflow

Rethinking this approach.  Not working so far.


Include meta in total progress. Closes #166

This includes meta download progress in total progress. Code inspired by dat-next.

@joehand what about this as a simple fix for now?

@joehand what about this as a simple fix for now?

Works for now probably. I'm gonna chase this bug next week first thing!
This still may show as 100% done even if it is not (sometimes I had to get to 150% for it to be 'done'). So as long as you aren't stopping anything with 100% - that was my issue with the CLI was I exited at 100% even though that wasn't finished.

I think what you're saying is also in the case where *Total isn't fully known yet it will mistakenly show 100%, right?
Yeah, we're not triggering anything once 100% is reached, it's purely for UI.

This isn't running for me:

looks like you need to reinstall dat-node and bundle

ah, no! @yoshuawuyts this is because your PR fixing dat-node changed the DB schema, you need to wipe your db

This has become part of #198, files referenced no longer exist - closing! :sparkles:


Edit profile page takes too long to load - 15 seconds + for Tibet

Thanks for hashing this out a bit more @tibetsprague. In careful look at the Edit Profile page it appears that calling fetchForCurrentUser is querying much more than the data needing refresh for the edit profile (i.e. notifications, etc--as you pointed out). The edit page could be made available much more quickly in that case by re-querying just the fields on Me which are being edited instead of using the full fetchForCurrentUser, and also potentially taking-out the pending load for savedSearches across the whole UserSettings area and pushing it down into a loading indicator on the Edit Saved Searches tab.
Copying this note onto a new ticket re. the loading time of Edit Profile.


Broken example file configure-db.sh (with fix)

The current version of file /linux/preview/examples/mssql-customize/configure-db.sh has multiple errors. The following version functions properly:
#!/bin/bash

# Wait 60 seconds for SQL Server to start up by ensuring that
# calling SQLCMD does not return an error code, which will ensure that sqlcmd is accessible
# and that system and user databases return "0" which means all databases are in an "online" state
# https://docs.microsoft.com/en-us/sql/relational-databases/system-catalog-views/sys-databases-transact-sql?view=sql-server-2017

DBSTATUS=1
ERRCODE=1
i=0

while [[ $DBSTATUS -ne 0 ]] && [[ $i -lt 60 ]] && [[ $ERRCODE -ne 0 ]]; do
	i=$(($i+1))
	DBSTATUS=$(/opt/mssql-tools/bin/sqlcmd -h -1 -t 1 -U sa -P $MSSQL_SA_PASSWORD -Q "SET NOCOUNT ON; select SUM(state) from sys.databases")
	ERRCODE=$?
	sleep 1
done

if [ $DBSTATUS -ne 0 ] || [ $ERRCODE -ne 0 ]; then
	echo "SQL Server took more than 60 seconds to start up or one or more databases are not in an ONLINE state"
	exit 1
fi

# Run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -U sa -P $MSSQL_SA_PASSWORD -d master -i setup.sql

I would add also that for the latest 2022 image I had to change also /opt/mssql-tools/bin/sqlcmd for /opt/mssql-tools18/bin/sqlcmd


Support for Cypress

Hi @jan-molak ,
Working on serenity-js, gives us power to customise our test scenarios and make it flexible and readable. So far best Framework. :)
Is it a better idea to use serenityJS- protractor in some other framework or
shall i integrate it with some other framework?
There is a framework available for us to work, where,
browser selection option on UI is available.
screenshots comes while run is going on and can see results.
Is there any way where we can see in between run result of any particular scenario in serenity js, if ran as a test suite(having more than 300 ) together.
Regards,
Abhineet Sharma

Hi @AbhineetSharmax and thanks for your kind words :-) I'm very happy to hear that Serenity/JS is helping you out!
It's difficult for me to advise without seeing the other framework you're referring to. If you could point me to its documentation or demos then I could try to suggest something.
Best,
Jan

Hi @jan-molak
Do you have any plans to integrate with cypress so we could use the best from both worlds?

@asnov - I think that the way Serenity/JS v2 is designed should make it possible to create an adapter for Cypress. I'm not sure how much time I will have to investigate that, but am open to ideas.

I've had quite a few people ask about Cypress recently, so if anyone would like to work with me on a PR to support it I'd be happy to help out.

Let's move this conversation to #493

Conversation moved to #846


Masa React fails to build with masa-sdk 2.0

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
        - add a fallback 'resolve.fallback: { "buffer": require.resolve("buffer/") }'
        - install 'buffer'
If you don't want to include a polyfill, you can use an empty module like this:
        resolve.fallback: { "buffer": false }
resolve 'buffer' in '/Users/sebastian/projects/masa/libs/masa-sdk/dist/browser'
  Parsed request is a module
  using description file: /Users/sebastian/projects/masa/libs/masa-sdk/dist/package.json (relative path: ./browser)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      /Users/sebastian/projects/masa/libs/masa-sdk/dist/browser/node_modules doesn't exist or is not a directory
      /Users/sebastian/projects/masa/libs/masa-sdk/dist/node_modules doesn't exist or is not a directory
      looking for modules in /Users/sebastian/projects/masa/libs/masa-sdk/node_modules
        single file module
          using description file: /Users/sebastian/projects/masa/libs/masa-sdk/package.json (relative path: ./node_modules/buffer)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Users/sebastian/projects/masa/libs/masa-sdk/node_modules/buffer doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Users/sebastian/projects/masa/libs/masa-sdk/node_modules/buffer.js doesn't exist
        /Users/sebastian/projects/masa/libs/masa-sdk/node_modules/buffer doesn't exist
      /Users/sebastian/projects/masa/libs/node_modules doesn't exist or is not a directory
      /Users/sebastian/projects/masa/node_modules doesn't exist or is not a directory

fixed


query rpc tracker - sort servers by key name

Hello,
I have created simple PR, currenly when I have multiple RPC server I can see then in random order, another issue is that ip address can have different length, e.g. <IP_ADDRESS> is 7, and <IP_ADDRESS> is 15, so the output may looks like:
Server List
----------------------------
server-address     key
----------------------------
<IP_ADDRESS>:5001 server:prague
<IP_ADDRESS>:5001 server:plex
<IP_ADDRESS>:5001        server:plex
<IP_ADDRESS>:5001 server:prague
<IP_ADDRESS>:5001        server:plex
<IP_ADDRESS>:5001 server:prague
<IP_ADDRESS>:5001        server:plex
----------------------------

after my changes:
Server List
------------------------------
server-address           key
------------------------------
     <IP_ADDRESS>:5001    server:plex
      <IP_ADDRESS>:5001    server:plex
     <IP_ADDRESS>:5001    server:plex
     <IP_ADDRESS>:5001    server:plex
      <IP_ADDRESS>:5001    server:prague
      <IP_ADDRESS>:5001    server:prague
      <IP_ADDRESS>:5001    server:prague
------------------------------

another issue, when I review client code, is that total_ct can be converted to defaultdict(int), and computing max_key_len can be shortened to if-condition-assignment


Supplier assessment behaviour

What does this pull request do?
Adds endpoint so that appointment behaviour can be submitted.
What is the intent behind these changes?
Behaviour needs to be able to be recorded against appointments for both action plan sessions & supplier assessments

Branched from supplier assessment pr branch(IC-1459) which should to go in first.

looks like this has a bunch of commits shared with your other PR. will hold off on review until the other is merged and this is rebased to avoid confusion


XOR branching/non-manifold result?

Hi,
Intersection, Union, and Difference work just fine, but I am having a hard time figuring out how I should read the output of a simple XOR clip as below. When I try to build a set of polycurves from the output results, my graphics library complains that the points form a branching/non-manifold result. Any help on how to correctly transform the data to a set of closed simple polygons would be appreciated.
import pyclipper
subj = (
((180, 200), (260, 200), (260, 150), (180, 150)),
((215, 160), (230, 190), (200, 190))
)
clip = ((190, 210), (240, 210), (240, 130), (190, 130))
pc = pyclipper.Pyclipper()
pc.AddPath(clip, pyclipper.PT_CLIP, True)
pc.AddPaths(subj, pyclipper.PT_SUBJECT, True)
solution = pc.Execute(pyclipper.CT_XOR, pyclipper.PFT_EVENODD, pyclipper.PFT_EVENODD)

If you want it to be closed polygons, the easiest thing is to append the first point onto the end of it. Add one more line solution.append(solution[0]))


Note added for community supported module

@melvingrootenboers Didn't know you were going to send a PR, I already documented this ;) Put it in another place, right under the App Store link for the module. Also fixed some note formatting. https://documentation-accp.cfapps.io/howto/extensibility/install-and-configure-the-smtp-module Will close this PR.


is it possible to somehow determine in which units of measurement the model is located? Millimeters, centimeters, meters or inches?

is it possible to somehow determine in which units of measurement the model is located? Millimeters, centimeters, meters or inches?

The easiest way to determine this is via https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#coordinate-system-and-units :

The units for all linear distances are meters.

Yes, creators of models can do whatever they want. (Even in the official sample models, there are some models that do not have a "reasonable" size in terms of their bounding box - see this issue for some examples). But one purpose of putting this information into the specification is that users generally should be able to rely on that, and do not have to deal with something like "asset": { "units": "nauticMiles" } } or so...


Cannot build my project

I'm trying to build my vite project using auth0 and react.
When I build I get the following error :
[commonjs--resolver] Unexpected token (58:108) in<EMAIL_ADDRESS>file: ...node_modules/@auth0/auth0-react/dist/auth0-react.esm.js:58:108
56:
57: function __awaiter(thisArg, _arguments, P, generator) {
58:     function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value) })}
                                                                                ^
59:     return new (P || (P = Promise))(function (resolve, reject) {
60:         function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
error during build:
SyntaxError: Unexpected token (58:108) in<EMAIL_ADDRESS>    at pp$4.raise (file://...node_modules/rollup/dist/es/shared/rollup.js:20533:13)
    at pp$9.unexpected (file://...node_modules/rollup/dist/es/shared/rollup.js:17834:8)
    at pp$5.parseIdent (file://...node_modules/rollup/dist/es/shared/rollup.js:20464:10)
    at pp$7.parseBindingAtom (file://...node_modules/rollup/dist/es/shared/rollup.js:19093:15)
    at pp$7.parseMaybeDefault (file://...node_modules/rollup/dist/es/shared/rollup.js:19128:23)
    at pp$5.parsePropertyValue (file://...node_modules/rollup/dist/es/shared/rollup.js:20212:35)
    at pp$5.parseProperty (file://...node_modules/rollup/dist/es/shared/rollup.js:20203:8)
    at pp$5.parseObj (file://...node_modules/rollup/dist/es/shared/rollup.js:20158:21)
    at pp$7.parseBindingAtom (file://...node_modules/rollup/dist/es/shared/rollup.js:19090:19)
    at pp$7.parseMaybeDefault (file://...node_modules/rollup/dist/es/shared/rollup.js:19128:23)

I am using auth0 "@auth0/auth0-react": "2.0.0" and vite "vite": "^4.1.0"

Hey @aleguern, I just tested using auth0-react in a freshly created react-ts template from create-vite and was able to run locally and build the project. Could you please provide a minimal sample that reproduces this issue so we can investigate if this is an issue on our end?

Closing due to inactivity


No, all pulls must go over https, and the index only responds to 443 for this.
Really strange the corporate proxy doesn't allow 443... I'm sorry there isn't much help we can do here.

Which is fine, but if I specify HTTPS_PROXY, shouldn't Docker respect this?


astroquery: init at 0.3.8

Motivation for this change
Package for the astroquery Python module for Nix.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build python36Packages.matplotlib

@GrahamcOfBorg build python36Packages.astroquery

@GrahamcOfBorg build python27Packages.astroquery

No attempt on x86_64-linux (full log)
The following builds were skipped because they don't evaluate on x86_64-linux: python27Packages.astroquery
Partial log (click to expand)
while evaluating anonymous function at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/stdenv/generic/make-derivation.nix:144:17, called from undefined position:
while evaluating 'callPackageWith' at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/lib/customisation.nix:108:35, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/top-level/python-packages.nix:197:13:
while evaluating 'makeOverridable' at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/lib/customisation.nix:67:24, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/lib/customisation.nix:112:8:
while evaluating anonymous function at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/development/python-modules/astropy/default.nix:1:1, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/lib/customisation.nix:69:12:
while evaluating 'makeOverridablePythonPackage' at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/top-level/python-packages.nix:40:37, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/development/python-modules/astropy/default.nix:8:1:
while evaluating 'makeOverridable' at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/lib/customisation.nix:67:24, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/top-level/python-packages.nix:42:12:
while evaluating anonymous function at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/development/interpreters/python/build-python-package.nix:27:1, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/lib/customisation.nix:69:12:
while evaluating anonymous function at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/development/interpreters/python/mk-python-derivation.nix:15:1, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/development/interpreters/python/build-python-package.nix:44:4:
astropy-3.0.5 not supported for interpreter python2.7

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: python27Packages.astroquery
Partial log (click to expand)
while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/stdenv/generic/make-derivation.nix:144:17, called from undefined position:
while evaluating 'callPackageWith' at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/lib/customisation.nix:108:35, called from /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/top-level/python-packages.nix:197:13:
while evaluating 'makeOverridable' at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/lib/customisation.nix:67:24, called from /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/lib/customisation.nix:112:8:
while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/development/python-modules/astropy/default.nix:1:1, called from /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/lib/customisation.nix:69:12:
while evaluating 'makeOverridablePythonPackage' at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/top-level/python-packages.nix:40:37, called from /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/development/python-modules/astropy/default.nix:8:1:
while evaluating 'makeOverridable' at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/lib/customisation.nix:67:24, called from /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/top-level/python-packages.nix:42:12:
while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/development/interpreters/python/build-python-package.nix:27:1, called from /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/lib/customisation.nix:69:12:
while evaluating anonymous function at /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/development/interpreters/python/mk-python-derivation.nix:15:1, called from /var/lib/gc-of-borg/nix-test-rs-6/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-6/pkgs/development/interpreters/python/build-python-package.nix:44:4:
astropy-3.0.5 not supported for interpreter python2.7

Success on x86_64-linux (full log)
Attempted: python36Packages.matplotlib
Partial log (click to expand)
copying path '/nix/store/v9g1wflx11c57cjx3qy2ysjiydfx46wh-python3.6-pytz-2018.5' from 'https://cache.nixos.org'...
copying path '/nix/store/92bpv9zz1v5pvp67jzgnqx19jr545jiz-python3.6-setuptools_scm-3.1.0' from 'https://cache.nixos.org'...
copying path '/nix/store/gh75sa1lkavznxqv9qbsbvqbkx4j79zs-python3.6-six-1.11.0' from 'https://cache.nixos.org'...
copying path '/nix/store/hdj7q1kna4ylsjhyg4ax2fykvvpnilbl-python3.6-cycler-0.10.0' from 'https://cache.nixos.org'...
copying path '/nix/store/kdh8smsqn4fll01ic4441pq23wr8hid9-python3.6-mock-2.0.0' from 'https://cache.nixos.org'...
copying path '/nix/store/gm0qfvbivrsqfvbfdb7igvp0qdlgl9r5-python3.6-python-dateutil-2.7.3' from 'https://cache.nixos.org'...
copying path '/nix/store/f7x23prfxfkh7212qbmbqh5azsrspykz-python3.6-singledispatch-<IP_ADDRESS>' from 'https://cache.nixos.org'...
copying path '/nix/store/q03rxjwp6kiaf4i4lp0dq1h6dnhn3d52-python3.6-tornado-5.1' from 'https://cache.nixos.org'...
copying path '/nix/store/mncdw6rx7da6mvjw17s7rx9qy19q1mj0-python3.6-matplotlib-3.0.2' from 'https://cache.nixos.org'...
/nix/store/mncdw6rx7da6mvjw17s7rx9qy19q1mj0-python3.6-matplotlib-3.0.2

Success on aarch64-linux (full log)
Attempted: python36Packages.astroquery
Partial log (click to expand)
Installing collected packages: astroquery
Successfully installed astroquery-0.3.8
/build/astroquery-0.3.8
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/bggiwk55x6yn8l2rlkl0zgyk2a0npqns-python3.6-astroquery-0.3.8
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/bggiwk55x6yn8l2rlkl0zgyk2a0npqns-python3.6-astroquery-0.3.8/lib
patching script interpreter paths in /nix/store/bggiwk55x6yn8l2rlkl0zgyk2a0npqns-python3.6-astroquery-0.3.8
checking for references to /build in /nix/store/bggiwk55x6yn8l2rlkl0zgyk2a0npqns-python3.6-astroquery-0.3.8...
pytestcachePhase

Success on x86_64-linux (full log)
Attempted: python36Packages.astroquery
Partial log (click to expand)
Successfully installed astroquery-0.3.8
/build/astroquery-0.3.8
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/wlf3n3lcckwl0a49g6g5kz9cmr8h9jdx-python3.6-astroquery-0.3.8
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/wlf3n3lcckwl0a49g6g5kz9cmr8h9jdx-python3.6-astroquery-0.3.8/lib
patching script interpreter paths in /nix/store/wlf3n3lcckwl0a49g6g5kz9cmr8h9jdx-python3.6-astroquery-0.3.8
checking for references to /build in /nix/store/wlf3n3lcckwl0a49g6g5kz9cmr8h9jdx-python3.6-astroquery-0.3.8...
pytestcachePhase
cannot link '/nix/store/.links/1rvdki2zf23625lnddixs8fli3926w0har0gw36lw0m9jnvsfb6s' to '/nix/store/wlf3n3lcckwl0a49g6g5kz9cmr8h9jdx-python3.6-astroquery-0.3.8/lib/python3.6/site-packages/astroquery/xmatch/core.py': No space left on device

Success on aarch64-linux (full log)
Attempted: python36Packages.matplotlib
Partial log (click to expand)
shrinking /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2/lib/python3.6/site-packages/matplotlib/_png.cpython-36m-aarch64-linux-gnu.so
shrinking /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2/lib/python3.6/site-packages/matplotlib/_qhull.cpython-36m-aarch64-linux-gnu.so
shrinking /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2/lib/python3.6/site-packages/matplotlib/_tri.cpython-36m-aarch64-linux-gnu.so
shrinking /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2/lib/python3.6/site-packages/matplotlib/ft2font.cpython-36m-aarch64-linux-gnu.so
shrinking /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2/lib/python3.6/site-packages/matplotlib/ttconv.cpython-36m-aarch64-linux-gnu.so
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2/lib
patching script interpreter paths in /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2
checking for references to /build in /nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2...
/nix/store/7mpxk499bwivs1b2yqnlwf85166fqz7y-python3.6-matplotlib-3.0.2


any port for c#

Hello sir
Any port for c# browser.vb, i still no clue make awesomium thread working.
Using awesomium 1.7.5
Thank you

sorry mate, i'm not fluent in c#.
you can try using sharp develop v4.4 to convert this project to c#.
if your requirement is same as mine (to get the final rendered html output of a web page), you don't need to convert this code to c#. you can add this project to your visual studio solution and reference it from your c# project and simply use it.
Test.vb of this project shows how to use it. simply instantiate the browser class and call the GetRenderedHTML method.

Wow, I didnt know sharp develop have convert ability. :) , previously I tried this http://converter.telerik.com/, but It have many error.
My goal is not just html, but automation and multiple scraping. This awesomium made my head spin off for 3 weeks. Maybe I should tried CEFsharp :)

yes .net 4.5.
cefsharp is not as polished as awesomoum imho.
i haven't done any automation as of yet, so can't help there.
but i found a code snippet which should give u an idea:
    void Html5_LoadingFrameComplete(object sender, Awesomium.Core.FrameEventArgs e)
    {

        // you can check url, do somthing before do this block.

        dynamic document = (Awesomium.Core.JSObject)Html5.ExecuteJavascriptWithResult("document");

        using (document)
        {
            var textbox = document.getElementsByTagName("input");

            if (textbox == null)
                return;

            for (int i = 0; i < textbox.length; i++ )
            {
                if (textbox[i].getAttribute("type") == "text")
                    textbox[i].value = "";

                if (textbox[i].getAttribute("type") == "password")
                    textbox[i].value = "";
            }
        }

    }

basically what you do is handle the LoadingFrameComplete event and execute javascript in order to interact with the elements on page.

thanks for the example. I already doing automation but when its coming  to running multiple step ( reusing
For example
Step 1. I want to get element from first website ( screenshot ).
Step 2. Then navigate to second website to post data.
Here my code with error comment :) , go to button1_click  ( its too long to paste here )
http://pastebin.com/m6RNiBeL
if Iam using Cefsharp its much simpler and easier
 static bool  ahb = true;
        private void button1_Click(object sender, EventArgs e)
        {
            button1.Enabled = false;

            const string testUrl = "http://www.mangahere.co/";

            // Create the offscreen Chromium browser.
            browser = new ChromiumWebBrowser(testUrl);

            // An event that is fired when the first page is finished loading.
            // This returns to us from another thread.
            browser.FrameLoadEnd += BrowserFrameLoadEnd;

            // We have to wait for something, otherwise the process will exit too soon.
            while (ahb)
            {
                Thread.Sleep(5000);

            }
            MessageBox.Show("Done step 1");

//========== here step2 ==========//

//========== end step2 ==========//

            // Clean up Chromium objects.  You need to call this in your application otherwise
            // you will get a crash when closing.
            Cef.Shutdown();

        }

i made a little console app for you to show how to get a screenshot of a page.
get it from here
it's in vb so you'll have to use a converter if you can't read vb.
you'll have to figure out the posting data to a form by yourself cause i dont have much free time these days.
good luck!

Thank mate for your effort. I still got error when webcore.dowork
 WebCore.DoWork(() =>
            {
                Session.Dispose();
                Session = null;
                return null;
            });
        }

error
Error	2	The type arguments for method 'Awesomium.Core.WebCore.DoWork<TResult>(System.Func<TResult>)' cannot be inferred from the usage. Try specifying the type arguments explicitly.	C:\Users\flowingair\Documents\Visual Studio 2010\Projects\AccGodMode\AccGodMode\Class1.cs	40	13	AccGodMode

SS

I give up, I trying new solution using CEFSHARP
thank for your time mate.

Here finally without error convert vb to c#, only Invoke on  view.source didnt work.( I comment it ). I dont know why. View.LoadingFrameComplete never fired.
C# .net 45 , I leave here in case anyone maybe have better knowledge about C# than me.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Diagnostics;
using Awesomium.Core;
using System.Threading.Tasks;
using System.Windows.Forms;

public class Class1 : IDisposable
{

    private static System.Threading.Thread Thread;
    private static bool CoreIsRunning = false;
    private static WebSession Session;

    WebView View;
    private static void AwesomiumThread()
    {
        if (WebCore.IsInitialized == false & CoreIsRunning == false)
        {
            WebCore.Initialize(new WebConfig { LogLevel = LogLevel.None }, false);
            WebCore.Run((s, e) => {
                CoreIsRunning = true;
            });
        }
    }

    public static void StartWebCore()
    {
        if (WebCore.IsInitialized == false && CoreIsRunning == false && (Thread == null))
        {
            Thread = new System.Threading.Thread(AwesomiumThread);
            Thread.Start();
        }
    }

    public static void StopWebCore()
    {
        if ((Session != null))
        {
            WebCore.DoWork(() =>
            {
                Session.Dispose();
                Session = null;
                return Session;
            });
        }
        WebCore.Shutdown();
        Thread = null;
        CoreIsRunning = false;
    }

    private static void InitSession()
    {
        if ((Session == null))
        {
            Session = WebCore.DoWork(() => { return WebCore.CreateWebSession(new WebPreferences { CanScriptsOpenWindows = false, DefaultEncoding = "utf-8", MaxHttpCacheStorage = 64000000 }); });
        }
    }

    private void CreateView()
    {
        if ((View != null))
        {
            View.LoadingFrameComplete -= null;
            WebCore.DoWork(() =>
            {
                if ((View != null))
                {
                    View.Dispose();
                }
                return View;
            });
            View = null;
        }
        View = WebCore.DoWork(() => { return WebCore.CreateWebView(1000, 1000, Session, WebViewType.Offscreen); });

    }

    public void Browser()
    {
        StartWebCore();
        InitSession();
        CreateView();
    }

    public void New()
    {
	    StartWebCore();
	    InitSession();
	    CreateView();
    }

    public void TakeScreenShot(string URL)
    {
        while (!(CoreIsRunning))
        {
            Task.Delay(100).Wait();
        }

        View.LoadingFrameComplete += (s, e) =>
        {
            if (e.IsMainFrame)
            {
                BitmapSurface surface = (BitmapSurface)View.Surface;
                surface.SaveToPNG("c:\\screenshot.png");
                //Interaction.MsgBox("Image saved!");
            }
        };

        //Here its doesnt WORK =======================//

        View.BeginInvoke(new Action(() => { View.Source = URL.ToUri(); }));

    }

    #region "IDisposable Support"
    private bool disposedValue;
    protected virtual void Dispose(bool disposing)
    {
        if (!this.disposedValue)
        {
            if (disposing)
            {
                View.LoadingFrameComplete -= null;
                WebCore.DoWork(() =>
                {
                    if ((View != null))
                    {
                        View.Dispose();
                        View = null;
                    }
                    return View;
                });
            }
        }
        this.disposedValue = true;
    }
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }
    #endregion

}

that's easy.
there's no BeginInvoke method on the view object.
change that line to:
View.Invoke(new Action(() => { View.Source = URL.ToUri(); }));
also you might have to change the save path to something other than c:\\ because i think dot net doesnt have permission to write to the C: drive.

this works for me on my computer:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Diagnostics;
using Awesomium.Core;
using System.Threading.Tasks;

public class ScreenShot : IDisposable
{

    private static System.Threading.Thread Thread;
    private static bool CoreIsRunning = false;
    private static WebSession Session;

    WebView View;
    private static void AwesomiumThread()
    {
        if (WebCore.IsInitialized == false & CoreIsRunning == false)
        {
            WebCore.Initialize(new WebConfig { LogLevel = LogLevel.None }, false);
            WebCore.Run((s, e) =>
            {
                CoreIsRunning = true;
            });
        }
    }

    public static void StartWebCore()
    {
        if (WebCore.IsInitialized == false && CoreIsRunning == false && (Thread == null))
        {
            Thread = new System.Threading.Thread(AwesomiumThread);
            Thread.Start();
        }
    }

    public static void StopWebCore()
    {
        if ((Session != null))
        {
            WebCore.DoWork(() =>
            {
                Session.Dispose();
                Session = null;
                return Session;
            });
        }
        WebCore.Shutdown();
        Thread = null;
        CoreIsRunning = false;
    }

    private static void InitSession()
    {
        if ((Session == null))
        {
            Session = WebCore.DoWork(() => { return WebCore.CreateWebSession(new WebPreferences { CanScriptsOpenWindows = false, DefaultEncoding = "utf-8", MaxHttpCacheStorage = 64000000 }); });
        }
    }

    private void CreateView()
    {
        if ((View != null))
        {
            View.LoadingFrameComplete -= null;
            WebCore.DoWork(() =>
            {
                if ((View != null))
                {
                    View.Dispose();
                }
                return View;
            });
            View = null;
        }
        View = WebCore.DoWork(() => { return WebCore.CreateWebView(1000, 1000, Session, WebViewType.Offscreen); });

    }

    public ScreenShot()
    {
        StartWebCore();
        InitSession();
        CreateView();
    }

    public void TakeScreenShot(string URL)
    {

        while (!(CoreIsRunning))
        {
            Task.Delay(100).Wait();
        }

        View.LoadingFrameComplete += (s, e) =>
        {
            if (e.IsMainFrame)
            {
                BitmapSurface surface = (BitmapSurface)View.Surface;
                surface.SaveToPNG("d:\\screenshot.png");
                Console.WriteLine("Image saved!");
            }
        };

        View.Invoke(new Action(() => { View.Source = URL.ToUri(); }));

    }

    #region "IDisposable Support"
    private bool disposedValue;
    protected virtual void Dispose(bool disposing)
    {
        if (!this.disposedValue)
        {
            if (disposing)
            {
                View.LoadingFrameComplete -= null;
                WebCore.DoWork(() =>
                {
                    if ((View != null))
                    {
                        View.Dispose();
                        View = null;
                    }
                    return View;
                });
            }
        }
        this.disposedValue = true;
    }
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }
    #endregion

}

Still dont work. Its this posible about GUI ( winform ) and console have different way ?
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using DeathByCaptcha;
using System.Net;
using System.IO;
using CsQuery;
using CsQuery.Web;
using System.Threading;
using Awesomium.Core;
using System.Threading.Tasks;
using RestSharp;
using System.Drawing.Imaging;
// Do not forget to reference DeathByCaptcha.dll in your project!
using DeathByCaptcha;
using Awesomium.Windows.Forms;
using Awesomium.Core.Data;

namespace AccGodMode
{
    public partial class MainAcc : Form
    {

        public MainAcc()
        {

            InitializeComponent();

            ScreenShot.StartWebCore();
        }
        static bool datate = true;

        private void button1_Click(object sender, EventArgs e)
        {
            button1.Enabled = false;

            using (ScreenShot browser = new ScreenShot())
            {
                browser.TakeScreenShot("http://google.com");

            }

            ScreenShot .StopWebCore();

        }

    }
}

here's a working forms project.
enjoy!

Wow that really worked out, Now I can run and reusing webview.
Many thanks for your time, you're very kind. :)
Care to explain the  magic behind this...  ( what wrong with my code, I still have no clue  )
And where this little code called?
    public ScreenShot()
    {
        StartWebCore();
        InitSession();
        CreateView();
    }

I didnt really understand where this part called. :)
Please enlighten me?

you're most welcome mate!
i just added this part:
    while (!ScreenShotTaken)
    {
        Task.Delay(100).Wait();
    }

we have to wait until the LoadingFrameComplete is fired. otherwise the method will return before the event is fired and the Screenshot object gets disposed.
and public ScreenShot() {...} is called when you instantiate the Screenshot class with the following:
using (ScreenShot browser = new ScreenShot() ) {...}
It's the c# constructor for the class.
good luck with your project...

Great website mate, its iis server with vb.net technology behind?

And thanks for the Enlightenment, sifu #bow

thanks!
yeah it's a vb.net mvc app running on iis as the front-end and a winforms app running as the scheduler.


records with no fulltext

Need to make sure the records with abstract only will be dealt with and entities in title/abstract identified correctly.
@aaccomazzi

Please make sure that keywords are considered as well.
There are a number of examples where we have the body but the target is missed, for example: https://www.scixplorer.org/search?q="tycho+crater"+NOT+gpn%3A"crater%2Ftycho"+has%3Abody+year%3A2000-2020&sort=score+desc&sort=date+desc&n=10&p=1
@sjarmak it would be good to know if there are other cases you've come across that are being missed.

@aaccomazzi from a quick look, there are many cases like this, e.g., gusev crater has 367 records (11 out of possible 378 tagged), gale crater 748 records (7 out of possible 755 tagged), copernicus crater 75 records (1 out of possible 76 tagged), mare imbrium 74 records (2 out of a possible 76 are tagged)


add method doesnt exists in presentation class

[Enter feedback here]
async function addSlide() {
await PowerPoint.run(async function(context) {
context.presentation.slides.add();
await context.sync();

});
}
this code doesnt work
error: Property 'add' does not exist on type 'SlideCollection'.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cb0c7511-1f0e-76f3-f274-531f13c77621
Version Independent ID: c305f14b-2b51-8b45-566b-9b33d630c8fd
Content: Add and delete slides in PowerPoint - Office Add-ins
Content Source: docs/powerpoint/add-slides.md
Product: powerpoint
Technology: add-ins
GitHub Login: @o365devx
Microsoft Alias: o365devx

Hello @amahmadnia,
Thank you for raising this issue. SlideCollection.add is currently is preview. To use that API, you'll need to follow the steps listed here. That will enable preview APIs in your add-in. Please let me know if that fixes your issue.
I'm also going to add the preview-enabling steps to the article you linked. Hopefully, that'll make it more clear.


Add additional documentation and examples for HTML rendering

There does not appear to be a lot of documentation for HTML rendering.
Need more documentation and ideally a repo that contains fully working examples of how to set up and use the HTML rendering functionality.

The response which contains the HTML code is just text. Any rendering is something you should test locally.


valdate project names in create

validate project names in create (fix https://github.com/flutter/flutter/issues/2283)

This in-lines our current deps.

LGTM modulo blacklist.

Can't we determine this list automatically from the .packages file of the flutter package?

Can't we determine this list automatically from the .packages file of the flutter package?

We haven't necessarily computed that on the end-developer's machine.  We used to need to compute it to get the engine revision, but now we store the engine revision explicitly in the repo.

  'args',
  'async',
  'collection',
  'convert',
  'flutter',
  'html',
  'intl',
  'logging',
  'matcher',
  'mime',
  'path',
  'plugin',
  'pool',
  'test',
  'utf',
  'watcher',
  'yaml'


[SPARK-35647][PYTHON][DOCS] Restructure User Guide in PySpark documentation

What changes were proposed in this pull request?
This PR proposes to restructure User Guide in PySpark documentation for pandas APIs on Spark.
Before

After

Note that I mostly just moved the contents around except minor changes:

Removing some questions in FAQ that don't make sense in Apache Spark
Rename a subtitle "Working with pandas and PySpark" to "From/to pandas and PySpark DataFrames"

For renaming Koalas to either pandas-on-Spark or pandas APIs on Spark, it will be done at SPARK-35591
Why are the changes needed?
For better readability.
Does this PR introduce any user-facing change?
Yes, it restructures the documentation as shown above.
How was this patch tested?
I manually built the docs and tested.

Test build #139479 has started for PR 32820 at commit 3fd1721.

Test build #139479 has finished for PR 32820 at commit 3fd1721.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44002/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139479/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44002/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44002/

LGTM! Thanks.

Thanks @xinrong-databricks.
Merged to master.