[Issue]: AWS Bedrock Function calling

Describe the issue
I am working on autogen using meta.llama3-8b-instruct-v1:0 from AWS bedrock. And i face an issue where the agents are not using the function. Whereas while working with groq, it works well with function calling. @marklysze Can you please give me some ideas to fix this issue?
Steps to reproduce
No response
Screenshots and logs
No response
Additional Information
No response

Hey @SumaiyaSultan2002, we should be merging in an Amazon Bedrock client soon, and that would be suitable to use. See #3232.
You could try out using that code if you wanted to.
Can you provide an example of the config settings (be mindful of removing keys and tokens) so we can see which client class it is using.

yes i have cloned and made use of the code.  I find that it is not making use of the function for the agents.
"aws_region_name": "us-east-1",
"api_type": "bedrock",
"model": "meta.llama3-8b-instruct-v1:0",
"aws_access_key": " ",
"aws_secret_key": " ",
"price" : [0.003, 0.015],

Thanks for that @SumaiyaSultan2002, just a note that the region parameter is aws_region not aws_region_name - there was an error in the comment for the Bedrock client class, apologies.
I tried the config and Bedrock returned:
An error occurred (ValidationException) when calling the Converse operation: This model doesn't support tool use.
It's a subtle model difference - Llama 3 doesn't support tools but Llama 3.1 does. Llama 3.1 is available in the us-west-2 AWS region. Model capabilities are here.
I've also found that the 8B model isn't as good at calling functions as the 70B model.
If you want to try Llama 3.1 70B in us-west-2 (the 3.1 70B model is cheaper than Llama 3 8B :) ):
        "api_type": "bedrock",
        "model": "meta.llama3-1-70b-instruct-v1:0",
        "aws_region": "us-west-2",
        "price": [0.00099, 0.00099],
        "cache_seed": None,  # turn off caching, optional
        "aws_access_key": "",
        "aws_secret_key": "",

I've also noticed that Llama 3.1 tends to pass numeric parameters as strings, in this demo code we have checked if the number parameter is a string and converted it to a float. This example is in the PR's documentation change.
import json
from typing import Literal, Annotated

import autogen

config_list_bedrock = [
    {
        "api_type": "bedrock",
        "model": "meta.llama3-1-70b-instruct-v1:0",
        "aws_region": "us-west-2",
        "price": [0.00099, 0.00099],
        "cache_seed": None,  # turn off caching, optional
        "aws_access_key": "",
        "aws_secret_key": "",
    }
]

# Create the agent and include examples of the function calling JSON in the prompt
# to help guide the model
chatbot = autogen.AssistantAgent(
    name="chatbot",
    system_message="""For currency exchange and weather forecasting tasks,
        only use the functions you have been provided with.
        Output only the word 'TERMINATE' when an answer has been provided.
        Use both tools together if you can.""",
    llm_config={
        "config_list": config_list_bedrock,
    },
)

user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    is_termination_msg=lambda x: x.get("content", "") and "TERMINATE" in x.get("content", ""),
    human_input_mode="NEVER",
    max_consecutive_auto_reply=2,
)

# Currency Exchange function

CurrencySymbol = Literal["USD", "EUR"]

# Define our function that we expect to call
def exchange_rate(base_currency: CurrencySymbol, quote_currency: CurrencySymbol) -> float:
    if base_currency == quote_currency:
        return 1.0
    elif base_currency == "USD" and quote_currency == "EUR":
        return 1 / 1.1
    elif base_currency == "EUR" and quote_currency == "USD":
        return 1.1
    else:
        raise ValueError(f"Unknown currencies {base_currency}, {quote_currency}")

# Register the function with the agent
@user_proxy.register_for_execution()
@chatbot.register_for_llm(description="Currency exchange calculator.")
def currency_calculator(
    base_amount: Annotated[float, "Amount of currency in base_currency, float values (no strings), e.g. 987.82"],
    base_currency: Annotated[CurrencySymbol, "Base currency"] = "USD",
    quote_currency: Annotated[CurrencySymbol, "Quote currency"] = "EUR",
) -> str:
    # If the amount is passed in as a string, e.g. "123.45", attempt to convert to a float
    if isinstance(base_amount, str):
        base_amount = float(base_amount)

    quote_amount = exchange_rate(base_currency, quote_currency) * base_amount
    return f"{format(quote_amount, '.2f')} {quote_currency}"

# Weather function
# Example function to make available to model
def get_current_weather(location, unit="fahrenheit"):
    """Get the weather for some location"""
    if "chicago" in location.lower():
        return json.dumps({"location": "Chicago", "temperature": "13", "unit": unit})
    elif "san francisco" in location.lower():
        return json.dumps({"location": "San Francisco", "temperature": "55", "unit": unit})
    elif "new york" in location.lower():
        return json.dumps({"location": "New York", "temperature": "11", "unit": unit})
    else:
        return json.dumps({"location": location, "temperature": "unknown"})

# Register the function with the agent
@user_proxy.register_for_execution()
@chatbot.register_for_llm(description="Weather forecast for US cities.")
def weather_forecast(
    location: Annotated[str, "City name"],
) -> str:
    weather_details = get_current_weather(location=location)
    weather = json.loads(weather_details)
    return f"{weather['location']} will be {weather['temperature']} degrees {weather['unit']}"

# start the conversation
res = user_proxy.initiate_chat(
    chatbot,
    message="What's the weather in New York and can you tell me how much is 123.45 EUR in USD so I can spend it on my holiday?",
    summary_method="reflection_with_llm",
)

print(res.summary["content"])

Let me know how you go with that.

Thank you so much for the detailed explanation

Thank you so much for the detailed explanation

No problem, if you do test it can you let us know how you went and we'll assist further or close the issue if all is okay.

Closing the issue. As it was rectified.


I want to create a RLHF backend/frontend for labelling<=>training<=>correcting error loop.

If anyone has any lead on this please let me know. also anyone want to collaborate on this direction please let me know.

Sound like active learning?
https://docs.heartex.com/guide/active_learning.html
https://www.youtube.com/watch?v=8EO4vOw1MZc

@makseq While active learning is good but RLHF is quite different than that becuase it implements Reignforcement Learning for optimization of the model. All in all if you know what is RLHF it is quite different than active learning.

Yes, I know, but I expect to see your workflow in LS to achieve it. Seems you need Accept/Reject actions for your annotations?

Yes the RLHF can be done in multiple ways. You can have yes no type or ranking type.

Basically what I propose is the have a generalized RLHF model that goes at the output side of any model and instead of having supervised training we can have unsupervised training that can be supervised by the reinforcement model.

Maybe this repo will be helpful for you: https://github.com/heartexlabs/label-studio-RLHF/

@makseq maybe it is a private repo. giving me 404 error

Sorry, could you please check this one:
https://github.com/heartexlabs/RLHF


PyCon US 2023

Schedule: https://us.pycon.org/2023/schedule/
Playlist: https://www.youtube.com/playlist?list=PL2Uw4_HvXqvY2zhJ9AMUa_Z6dtMGF3gtb

I also noticed that PyCon US 2022 was missing, so I opened #1126 for that and noted the steps that I think would be necessary for getting this data from YouTube.

https://us.pycon.org/2023/schedule/conference.json might be helpful :)


Update minimal example to close #301

closes #301

@jph00 addressed your feedback in this PR (merging).


Jupyter extension in VSCode incorrectly alerting about file name overriding

Applies To

[ ] Notebooks (.ipynb files)
[X] Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?
With the Jupyter extension enabled, every file is marked as potentially overriding a Python package, even my test one "crap.py"
The message issued is:
This file could potentially override an existing Python package and interfere with kernel execution, consider renaming it.

Every single file in my directory is marked as such. This doesn't appear to affect .ipynb files
I noticed this appeared a week or two ago with the pre-release one but i never got around to diagnosing it. Now it's in the release version too
VS Code Version
Version: 1.70.0-insider (Universal) Commit: da76f93349a72022ca4670c1b84860304616aaa2 Date: 2022-08-03T05:55:27.651Z Electron: 18.3.5 Chromium: 100.0.4896.160 Node.js: 16.13.2 V8: <IP_ADDRESS>-electron.0 OS: Darwin x64 21.6.0
Jupyter Extension Version
v2022.8.1002161046
Jupyter logs
Visual Studio Code - Insiders (1.70.0-insider, undefined, desktop)
Jupyter Extension Version: 2022.8.1002161046.
Python Extension Version: 2022.13.12151008.
Workspace folder /Users/wardy/Documents/EMC/gitlab.nosync/storage
info 8:30:30.112: ZMQ install verified.
User belongs to experiment group 'jupyterTestcf'
User belongs to experiment group 'jupyterEnhancedDataViewer'
info 8:30:30.795: LSP Notebooks experiment is enabled
info 8:30:31.159: Experiment status for python is {"enabled":true,"optInto":[],"optOutFrom":[]}
info 8:30:32.182: Process Execution: > ~/Library/Caches/pypoetry/virtualenvs/storage-2bDQhgz_-py3.9/bin/python -m pip list
> ~/Library/Caches/pypoetry/virtualenvs/storage-2bDQhgz_-py3.9/bin/python -m pip list

Coding Language and Runtime Version
Python 3.9.0
Language Extension Version (if applicable)
python - v2022.13.12151008
pylance - v2022.7.40
Anaconda Version (if applicable)
not using
Running Jupyter locally or remotely?
locally

Duplicate of https://github.com/microsoft/vscode-jupyter/issues/11040
Will be fixed as a hot fix.


Image Generation

Hi
Using this code, I could sample text successfully ( model.sample()) but the image part is empty. Do you know how to sample (or generate) an image for the inference? It would be helpful if you could give me any pointers for this. Thank you!

@taeinkwon hey Taein, thank you for your interest
did you train with both text and images before sampling?

Yes I trained with both text and images like the following:
text_and_images = [[text, image] for text, image in zip(tokens['input_ids'], imgs)]
loss = model(text_and_images)

Do you have an example for generating images?

@taeinkwon could it be you included the eos id within your text? it needs to be placed after the image

@taeinkwon let me debug it tomorrow morning with a toy example

I am using BertTokenizer, and if I understand correctly, 101 is BOS and 102 is EOS which are included here.

I also replaced the BOS and EOS tokens as num_text_tokens and (num_text_tokens + 1) like written in
self.sos_id, self.eos_id = num_text_tokens, (num_text_tokens + 1)
but still the image output is None. Do you have any clues?

how long are your sequences?

It's very short. A few words. Now, I have fixed the length to 16.
I can't see any messages In the latest version.
What I found is that after training a few iterations, it can't generate images. However, when creating an initialized model, I can generate images with very small (ex., 8) num_text_tokens. I guess the training process breaks it?
I have attached my model FYI.
def create_model(vocab_size, image_size):
model = Transfusion(
num_text_tokens=vocab_size,
dim_latent=384,
channel_first_latent=True,
modality_default_shape=((image_size // 32, image_size // 32),),
modality_encoder=torch.nn.Sequential(
torch.nn.Conv2d(3, 64, kernel_size=3, stride=2, padding=1),
torch.nn.ReLU(),
torch.nn.Conv2d(64, 128, kernel_size=3, stride=2, padding=1),
torch.nn.ReLU(),
torch.nn.Conv2d(128, 256, kernel_size=3, stride=2, padding=1),
torch.nn.ReLU(),
torch.nn.Conv2d(256, 384, kernel_size=3, stride=1, padding=1),
),
modality_decoder=torch.nn.Sequential(
torch.nn.ConvTranspose2d(384, 256, kernel_size=3, stride=2, padding=1, output_padding=1),
torch.nn.ReLU(),
torch.nn.ConvTranspose2d(256, 128, kernel_size=3, stride=2, padding=1, output_padding=1),
torch.nn.ReLU(),
torch.nn.ConvTranspose2d(128, 64, kernel_size=3, stride=2, padding=1, output_padding=1),
torch.nn.ReLU(),
torch.nn.Conv2d(64, 3, kernel_size=3, stride=1, padding=1),
torch.nn.Sigmoid(),
),
transformer=dict(
dim=512,
depth=8
)
)
for param in model.parameters():
param.requires_grad = True
return model
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
tokenizer = AutoTokenizer.from_pretrained('Qwen/Qwen2.5-0.5B-Instruct')
image_size = 256
model = create_model(vocab_size=tokenizer.vocab_size, image_size=image_size)

@taeinkwon hey Taein, thank you for this, i will get a working demo using cifar-100 by end of today or tomorrow
this repository is being prioritized given a new robotics work based on Transfusion

Hi @lucidrains, I wonder whether you have figured out why images are not generated. I also checked this one https://github.com/lucidrains/transfusion-pytorch/discussions/14, and this code also can not generate images FYI.  Thank you!

@taeinkwon hey Taein, will take a look at this in the next few hours
thank you for your patience

@taeinkwon hey Taein, i found the issue and it is resolved in 0.3.5
when i added the ability for the language model to produce the length of the modality, i did not update the offset, leading to erroneous masking in adaptive modules as well as the text label
you can try running python train_toy.py and it should overfit on a small contrived example

@taeinkwon working?

Hi @lucidrains , I appreciate your update. I can check it this weekend. I will let you know whether it works or not by that time. Thank you!

Hi @lucidrains,
I trained image and text pairs for one hour. After that, the sample method only generates text without images. At least the text looks reasonable.
When I feed text to the model, I obtain an image that displays a grey color only. Somehow, training images do not work for me properly. I double-checked the inputs to the model, and the inputs look good to me. Do you have any ideas, or is there any place I need to check more?
I have also attached what the generated image's values look like.

@taeinkwon, did you expand the pixel values back out to int8 0-255?

@taeinkwon ah ok! so it is generating an image, but the image is grey, is that correct? I'll take another look tomorrow morning

More specifically, sampling does not work for images, and when I feed text, the image is grey. and yes I expanded the pixel values back out to int 8. Thank you for your investigation!

@taeinkwon hmm, does the toy example run for you?

Yes, the toy example works, for random numbers in the toy example, it does not converge to the specific numbers.
Is this repo tested with real images or achieved numbers in the benchmark of the original paper?
It seems like it happens when we have real examples. I used real images and text pairs for the training. FYI, the training code is very similar to the one in the discussion: https://github.com/lucidrains/transfusion-pytorch/discussions/14

@taeinkwon ok gotcha
no I have not confirmed it just yet, like noted in the todo, will be testing with Oxford flowers next

there is also an engineer at Nvidia who is interested in Transfusion and will be working on this soon

I have tried training this using MNIST digits for a few hours, and my observations are:

The training part looks good, flow loss down to 0.35 after an hour, does significantly denoise the partially noised sample if pred_flow is added to the noised sample.
Sample transitioning to image decoding does work after a few hundred steps, producing a 1x28x28 image which can be saved.
BUT the sample images themselves are pure noise, no discernable features

I'd suggest the problem lies in the odeint function.

@RefractAI thank you anon, both for this and your previous QA
I'll get it working, I'm fairly confident it will fly given pi-zero now

@taeinkwon @RefractAI i was able to get image modality only training working this morning, placed in train_image_only.py in the root folder
should be able to debug the issue by tomorrow 🤞 thank you for your patience

@taeinkwon mnist example at 5000 steps, with the language model asking for an image of zero (0)
i think it is fixed

Hi @lucidrains , thanks for your update. I checked the new update and the code you provided works. I also tried it with my dataset, and with the current resolution, it works, even with color images. I attached the generated image after 6 epochs.

However, some parts are still not resolved. I wanted to use the pretrained VAE model like the original TransFusion paper (see the below code), and some parts are not working.
vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse")
encoder = vae.encoder
decoder = vae.decoder
model = Transfusion(
        num_text_tokens=vocab_size,
        dim_latent=16,
        modality_default_shape=(( 128,128),),
        modality_encoder = encoder,
        modality_decoder = decoder,
        add_pos_emb = True,
        modality_num_dim = 2,
        transformer = dict(
        dim = 512,
        depth = 8,
    )
    ).cuda()

It seems like the decoder code does not work. It seems like this code does not call the decoder part for every iteration. The code only calls the decoder function when sampling. Also, I am not sure of the dimensionality when the model is sampling. Shouldn't it follow the dim and dim_latent in the Transfusion class?

If I put the end token for text when I train, the code can not sample images anymore. It only samples text. Without the end token, it works like the mnist example.

tokens[tokens==102] = self.tokenizer.vocab_size+1

@taeinkwon hi Taein, thank you for confirming the issue is resolved
for 1, unless I misunderstand, why would you require decoder to be called during training? are you not doing latent denoising, in which case only encoder is called during training and decoder called during sampling? (assuming pretrained vae)
for 2, this framework automatically takes care of the start and end text tokens for you. If you insert these manually, sampling will terminate where you place them. I assume you must be placing the end text token before the image in the list

@taeinkwon I have an open issue for VAE, and will get around to an end to end example tomorrow morning, smoothing out any issues in the process

Hi @lucidrains , thank you for your answer, for 2, now it is clear.
For 1, maybe I don't fully understand the paper, but I thought both VAE encoder and decoder are part of the training. The loss is calculated between the decoded image after latent denoising, isn't it? or at least U-net is part of the training process, which I believe we need to put in the modality_encoder and modality_decoder with the current framework.  (BTW, where can we put U-net, or is it already integrated into the code?)
Maybe I misunderstood it and the optimization only happens in the latent space.
Anyways, the example of the current output is like the following when I put the pretrained encoder and decoder:

The output dimensionality is also weird, 2048 x2048. I changed some parts of the code to fit batch inputs as the pretrained VAE (from the diffusers library) accepts batch inputs. My changes are:
Added L1414
modality = rearrange(modality, 'h w (p1 p2 c) -> c (h p1) (w p2)',c=4, p1 = 2, p2 = 2, h = 128)
modality=modality.unsqueeze(0)
Added L1818
modality_tensor = modality_tensor[0]

@taeinkwon Normally the VAE is frozen and not trained. The Transfusion paper doesn't train it. I'm just encoding and decoding using an external VAE myself and using Transfusion only with the pre-encoded latents.
Regarding the Unet up/down, it looks like it's on the todo list, but it's only for patchifying so there's no need for it if you use patch size 1.

Thanks for the clarification. For VAE, now I understand and will follow another open issue.
The reason why I ask about Unet is that I am also planning to put other modalities (such as audio) and wonder how I can put encoder decoder layers in that case to generate other signals. I probably need to change the code inside in that case?

@taeinkwon hey Taein, yes RefractAI is correct here
however, this discussion has spurred a thought, perhaps i can also involve the decoder with an extra reconstruction loss and perhaps even make it finetuneable. i'll try to incorporate that near end of week once i get the VAE and Unet training examples finished

@taeinkwon for more than one modality, there is an example in the tests. i understand Zimmerman's research (and yours by proxy) is still the audio / video self supervised learning
if you had to ask though, it probably means it isn't intuitive enough just yet. i'll work on making it until you are satisfied

@taeinkwon i'll get the unet in there tomorrow morning

Hi, I also tried to use pretrained VAE (wo u-net) after your update. I trained around 3~4 hours and the result is like the following:

I might need to train longer or debug it to find what the issue is. FYI, I checked the decoder and encoder and it reconstruct image well. (vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse"))
If you have any pointers where I should look at more, that would be helpful. Thanks!

@taeinkwon yea, that looks functional! you'll need to train longer / bigger transformer for sure

@taeinkwon i'll add the ability to fine tune given encoder / decoder on reconstruction loss by end of week

@taeinkwon what is AutoencoderKL, is it a huggingface thing?

https://github.com/huggingface/diffusers/blob/99c0483b67427de467f11aa35d54678fd36a7ea2/src/diffusers/models/autoencoders/autoencoder_kl.py#L262

@taeinkwon thanks! i'll play around with some oxford flowers latent flow matching later this week

@taeinkwon apologies Taein, could you give me one more day? had to refactor a bunch of things to get ready for the unet encoder / decoder
maybe i'll get a second wind this evening and finish it

One update from my side: my pretrained-VAE does not work even after two days of training. I need to investigate the problem more. Please let me know if you make the pretrained-VAE work for images (I guess Oxford flowers?). Thank you!

@taeinkwon hey Taein, sorry didn't get around to the unet this morning as, was caught up in some experiments of my own (always time consuming)
the VAE is definitely functional given the mnist vae example, but i'll get around to using the pretrained one from stable diffusion for oxford

One update from my side: my pretrained-VAE does not work even after two days of training. I need to investigate the problem more. Please let me know if you make the pretrained-VAE work for images (I guess Oxford flowers?). Thank you!

Diffusion models take hundreds of epochs to converge. Also, your model may be too small.
If you find an image model that does converge in less time than that, I would be very interested.

hey Taein, finally got the unet functionality completed, and will add an Oxford flowers example tomorrow morning

1 at 13k steps with contrived down and upsampling
i'll port in the full resnet blocks from a unet later this week

ran into some issue with the safe tensors package this morning when trying to load the pretrained vae
will try again later this evening
the file i'm working with lives here

@taeinkwon hey Taein, finally completed the script and i'm seeing results within the first 200 steps. do you want to take a look at what i have and see if it makes sense?

400 steps for modality only training
this doesn't rule out an issue with interspersed text training however, i'll get an example in there tomorrow morning using the labels that come with the oxford flower dataset

this is also using an additional reconstruction loss, which I don't have built for the main forward function yet, but will tomorrow

@lucidrains , thank you for sharing this result. Yes, I tested the code and I can't get a similar quality as I believe I don't have the additional reconstruction loss. I will test it with my dataset once you update it. Thank you!

@taeinkwon could you first try the modality only training to make sure the issue is with the lack of reconstruction loss? I'll expedite getting that in there tomorrow if so (for multimodal forward)

So when I ran the same code, train_latent_only.py, I got a similar result.
I am also testing a single modality (only images) with the EPIC-KITCHENS dataset, and the results are like the below after 4000 iters.  Maybe because the images are more complicated compared to the flower dataset. I will let the model train for a longer time.

One question: what is the purpose of L121-123 in the train_latent_only.py? I don't understand these lines.

@taeinkwon yup, i think you just need to leave it on overnight
as for L121-123, that is the logic for gradient accumulation (for bigger effective batch size)

~30k steps
this is with only 8k images, batch size 16
more training with more data Taein!

Hi @lucidrains , the result looks super good!
What I am trying now is

whether the result is still good for another dataset (Epic Kitchens) and
it can learn labels together (Oxford Flower).

So far (10k), I trained the images and labels together on the Oxford flower dataset and the label does not match.  Have you tried to learn labels together with the image reconstruction? I wonder what the results look like from your side. I will also share my results with another dataset once it is trained. Thank you!

@taeinkwon

"moon orchid" at 8.3k steps. i think i can see the orchid kind of?

ok, that one may be a fluke, i'll let it cook for a bit longer
learning a lot about flowers, as i need to google image each label to really know if it is correct 🤣

@taeinkwon so while it isn't perfect yet, at around 30k steps, it starts to get at least the color of the flowers correct
snapdragon

poinsettia

lotus

great masterwort (google imaging shows roughly the same color)

i'm sure with more training and data it would perform even better. ok i'm stopping the run early

https://arxiv.org/abs/2411.19722
Thought I'd highlight an interesting new text-image transformer paper, "Jetformer" from Deepmind. Takes an autoregressive approach with normalizing flows.

@RefractAI nice, i don't know if i believe in normalizing flows anymore, but i'll queue up the paper for future reading, thanks!

@RefractAI ahh, paper looks quite good actually, will read it this week :+1:

@RefractAI may pass on it, as the images in the appendix do not look that great. you can see lack of global coherence here and there, and the lemurs have a weird saturated color

@RefractAI think i will pass on the jetformer paper. bidirectional attention is still the way to go for modalities and that is where Transfusion shines

the most interesting part of their paper may end up being the noise curriculum

Yes, the noise curriculum part could be used for diffusion or flow too as it's just an augmentation.

@RefractAI curriculum learning is just underused in neural network training in general


i18n(de): update basics/astro-syntax.mdx

Description (required)

update basics/astro-syntax.mdx
#5364
#5572

Related issues & labels (optional)

Closes #
Suggested label:

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

de
basics/astro-syntax.mdx
Localization changed, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


Add a conda recipe on conda-forge

This looks like the simplest approach... https://conda-forge.org/docs/maintainer/adding_pkgs.html#the-staging-process

PR here https://github.com/conda-forge/staged-recipes/pull/24075

Done! https://github.com/conda-forge/redflag-feedstock
See also https://anaconda.org/conda-forge/redflag


Add JSON code formatting

And fix a typo

As suggested in https://github.com/activeloopai/Hub/pull/136#issuecomment-716210732, combined with https://github.com/activeloopai/Hub/pull/135.


Workaround for Numdifftools SciPy incompatibility

SciPy 1.3.0 breaks the latest Numdifftools release. A new release does not seem to be coming anytime soon because the maintainer is not responsive:
https://github.com/pbrod/numdifftools/issues/42
@lcnature, we only use Numdifftools in the BRSA tests. Any chance you could replace directionaldiff with something else? If that is not possible in the next few days, we will restrict SciPy to <1.3.0.
See also PyMC3's decision to remove it:
https://github.com/pymc-devs/pymc3/pull/3485

@lcnature, please do not spend too much time thinking about replacing directionaldiff if it is not simple. We may need to restrict SciPy to <1.3.0 anyway because of Pymanopt:
https://github.com/pymanopt/pymanopt/issues/77

Not necessary anymore after the latest Numdifftools release.


Update LoadingIconElement.java

Your pull request could not be accepted because

the result would be an icon that doesn't rotate
you are using an outdated version of Minecraft that ClickCrystals no longer supports (most likely earlier than 1.19.4)


cephfs: add ReadConfigFile implementing ceph_conf_read_file

Add a ReadConfigFile function that accepts a path to a config file. This will be especially handy when we need to connect to multiple ceph clusters from one (go-ceph test) function.
Checklist

[x] Added tests for features and functional changes
[x] Public functions and types are documented
[x] Standard formatting is applied to Go code

@nixpanic or @agarwal-mudit PTAL when you have a moment. Other PRs depend on this work.


Added managementapp-add. Solves #2917

Added managementapp-add. Solves #2917

@waldekmastykarz sorry for the confusion again ;-) but this PR looks better (reverted mkdocs.yml and it does show proper changes now)

Awesome! Thank you for cleaning it up. Hopefully it will help us get it merged 👏

As discussed in #3028 added a new PowerPlatformCommand

Merged manually. Thank you! 👏


Add Prompts + specs

Add Prompts
This PR adds the basic implementation for creating and managing prompt templates.

A prompt template refers to a reproducible way to generate a prompt. It contains a text string (“the template”), that can take in a set of parameters from the end user and generate a prompt. The prompt template may contain:
instructions to the language model; a set of few shot examples to help the language model generate a better response; a question to the language model.

Usage
An example prompt with one input variable:
prompt = Prompts::PromptTemplate.new(template: "Tell me a {adjective} joke.", input_variables: ["adjective"])
prompt.format(adjective: "funny") # "Tell me a funny joke."

An example prompt with multiple input variables:
prompt = Prompts::PromptTemplate.new(template: "Tell me a {adjective} joke about {content}.", input_variables: ["adjective", "content"])
prompt.format(adjective: "funny", content: "chickens") # "Tell me a funny joke about chickens."

Creating a PromptTemplate using just a prompt and no input_variables:
prompt = Prompts::PromptTemplate.from_template("Tell me a {adjective} joke about {content}.")
prompt.input_variables # ["adjective", "content"]
prompt.format(adjective: "funny", content: "chickens") # "Tell me a funny joke about chickens."

An example using few shot examples:
prompt = Prompts::FewShotPromptTemplate.new(
  prefix: "Write antonyms for the following words.",
  suffix: "Input: {adjective}\nOutput:",
  example_prompt: Prompts::PromptTemplate.new(
    input_variables: ["input", "output"],
    template: "Input: {input}\nOutput: {output}"
  ),
  examples: [
    { "input": "happy", "output": "sad" },
    { "input": "tall", "output": "short" }
  ],
   input_variables: ["adjective"]
)

prompt.format(adjective: "good")

# Write antonyms for the following words.
#
# Input: happy
# Output: sad
#
# Input: tall
# Output: short
#
# Input: good
# Output:

Loading a new prompt template using a JSON file:
prompt = Prompts.load_from_path(file_path: "spec/fixtures/prompts/prompt_template.json")
prompt.input_variables # ["adjective", "content"]
prompt = Prompts.load_from_path(file_path: "spec/fixtures/prompts/few_shot_prompt_template.json")
prompt.prefix #"Write antonyms for the following words."

@rafaelqfigueiredo If we can implement this prompt to use a JSON (or static) template using the new Prompts::FewShotPromptTemplate -- that would be amazing!

@rafaelqfigueiredo Could you please add this PR's description (Usage) to the README and also add a few examples to the examples/ dir?

@rafaelqfigueiredo Lastly -- could you please add RDoc-style code comments for the new code?


Add tests for json field string

Codecov Report

Merging #756 into master will increase coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #756      +/-   ##
==========================================
+ Coverage   51.29%   51.38%   +0.09%
==========================================
  Files          38       38
  Lines        8529     8529
==========================================
+ Hits         4375     4383       +8
+ Misses       3672     3661      -11
- Partials      482      485       +3

Impacted Files
Coverage Δ

xorm.go
66.19% <0%> (-1.41%)
:arrow_down:

session.go
55.05% <0%> (+1.49%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fdbf36d...fe03aaa. Read the comment docs.


Cleanup for Workbench internal code

📝 Description
Begins to clean up the code related to crafting and the workbench.
🎯 Goals
Make the code more readable and optimize the code to make it somewhat more efficient.
The following Base Files for stations:

[x] BlockStation
[x] GUIContainerStation
[x] TileEntityStation
[x] CraftingStation (/mwc/groovyscript/script/CraftingStation.java)

The following Workbench Specific Files:

[x] ContainerWorkbench
[x] CustomSearchTextField
[x] GUIButtonCustom
[x] GUIContainerWorkbench
[x] PagesSlot
[x] PagedSlotItemHandler
[x] TESRWorkbench
[x] WorkbenchBlock

❌ Non Goals
Adding new features, editing the UI of the workbenches, or any of the front-end.
🚦 Testing
Ran the mod after editing every file and followed the following:

Try craft gun with empty workbench.
Try craft gun with all items but insufficient.
Try craft gun with one item but sufficient quantity.
Try craft gun with sufficient items (all within one stack).
Try craft gun with sufficient items (all scattered between stacks).
Mixture of 4 and 5.
Try craft more than 1 at a time.

⏮ Backwards Compatibility
Since this is a workbench PR, this will be perfectly backwards compatible.
There is nothing here that can cause issues with updating. No weapons have been added/removed.
📚 Related Issues & Documents
N/A
📖 Added to documentation?

[ ] 📜 README.md
[ ] 📑 Documentation
[ ] 📓 Javadoc
[ ] 🍕 Comments
[x] 🙅 No documentation needed

Summary by CodeRabbit

New Features

Updated the positioning of GUI elements to enhance user interface experience.
Improved the search functionality within the crafting interface for better usability.
Enhanced crafting information tooltips for clarity and detail.

Bug Fixes

Added null check to prevent potential crashes during block harvesting.
Fixed typo in variable naming for material availability checks.
Corrected text input handling in the search field to improve filtering of crafting list results.

Refactor

Improved readability and maintainability of various methods across the crafting system.
Shifted towards immutability in class fields to ensure stability and predictability.
Restructured internal logic for crafting timer and progress tracking for better performance.

Style

Conducted minor formatting adjustments for a more consistent codebase appearance.
Removed redundant code comments and unnecessary line breaks to clean up the code.

Documentation

Updated error messages and comments to better guide users through crafting processes.

Chores

Renamed methods and variables to adhere to naming conventions and improve code clarity.

Testing - Please do not review just yet.


Debug Mode

Hi,
Thanks for sharing these functions!
Is there a way to enable a Debug mode and obtain a more detailed error logs ? Some of my 'custom' functions are failing and I am not able to see the issue in the console.
Example:
2018-08-22T23:50:10  Welcome, you are now connected to log-streaming service.
2018-08-22T23:51:10  No new trace in the past 1 min(s).
2018-08-22T23:52:10  No new trace in the past 2 min(s).
2018-08-22T23:52:57.870 [Info] Function started (Id=47be3062-4997-49c1-8277-5449b549a927)
2018-08-22T23:52:57.917 [Error] Function completed (Failure, Id=47be3062-4997-49c1-8277-5449b549a927, Duration=51ms)
Thanks,

An Error like that just means that the function threw an exception. You can add log output messages to your code to see more details about the cause.


docker

File: engine/reference/commandline/login.md

I suspect this was opened accidentally, or you forgot to describe the issue you're having. Closing this ticket, but feel free to continue the conversation

I suspect this was opened accidentally, or you forgot to describe the issue you're having. Closing this ticket, but feel free to continue the conversation


There's no coordinates information for France

First of all, I would like to thank you for providing the data.
I think it's the best quality of all.
There's no coordinates information for France. Please check.
And there is an area in China with the same code.
Shaanxi(SN), Shanxi(SN)
I'd appreciate it if you could check this as well.
It would be even better if you could let us know the update time by data source.

Thanks for the kind words and for flagging these issues.

There's no coordinates information for France. Please check.

I just added France's provinces, but didn't have time to fill the metadata which is a manual process.

And there is an area in China with the same code.
Shaanxi(SN), Shanxi(SN)

I had not noticed the error with Sha(a?)nxi, indeed there is a duplicate code but they are distinct provinces.
Both errors with France and China metadata should be easy enough to fix!

It would be even better if you could let us know the update time by data source.

This could be interpreted in a number of ways, I'm assuming you are interested in the "freshness" of the data:

The time at which the event described in the record occurred -- almost impossible to know for some sources, some have a delay of 1+ day and often even longer
The time at which a data source reported a record -- some report a daily snapshot at e.g. 10 AM CET, but some regularly report the entire history including fixes to old data which means that old records might get recent update times
The time at which a record was last updated in data.csv -- maybe useful for auditing purposes, but doesn't tell us much about when the data was actually collected and whether it's current
The last time that data.csv was updated -- also useful for auditing purposes, but seems like a bad proxy metric to measure the "freshness" of the data

To make things more complicated, there are a number of issues that have to be accounted for:

Some records may be updated due to new or fixed metadata (like Shaanxi region code) but the time at which the source reported the record is unchanged -- should the update time be updated then?
Some data sources used to have a large reporting delay but now are much faster, for example Spain -- we currently deliberately add days to the Date column to make sure we keep consistency in the data and to match the ECDC report, should the update time be in the future for an instance like this?
Some sources do not report when a record was added or updated, for example FR regional data -- should we use different definition of update time for those cases?

Can you help me understand how you are trying to use an "UpdateTime" column, if we were to add one?

I think the "freshness" of the data is sufficient.
 :)

I too would like to know when the data has been updated.
I am developing a web graphing tool and do not want to keep downloading data if it has not changed.
Currently I am using PHP and get_headers to check "Content-Length" and only load the file if that has changed.
It would be ok for my purpose to have the "UpdateTime" in the headers, but those without access to the file headers do not benefit. A separate file named 'UpdateTime.txt' would be good.

@Mahks you can get the last date that a commit was pushed to the repo using GitHub's API, see this for reference.
@quixote79 let me see if I can add a column "LastUpdated" which will represent when I last fetched the data for that particular row. So, for most rows, it will be updated every time since I'm retrieving historical data daily hoping to catch any potential corrections.

@quixote79 I looked into this, but I ran into a major problem. Currently, even though we are loading all historical data for almost countries, only a few rows get updated so you can visually inspect the changes. For example, look at this commit and click "Load diff".
By adding a LastUpdated column, almost every single row gets updated each time. Then the diff of the file would become useless and it's much harder to keep track of potentially bad changes to the whole file. I would have to make major changes and merge the current data with the previous days' data every time, which defeats the purpose of retrieving historical data.
As a proxy, you can look at when the repo was last updated to tell the approximate "freshness" of the data. Currently, only 2 types of data are not being updated every time the repo gets updated: Spain and Italy country-level data for dates prior to March.
To see when the last update was done to the repo, you can use GitHub's API which outputs a JSON file: https://api.github.com/repos/open-covid-19/data/commits. The date will be the "freshness" time:
$.getJSON('https://api.github.com/repos/open-covid-19/data/commits', data => {
    const lastUpdated = data[0].commit.committer.date;
}

Hopefully this is sufficient for your purposes, let me know if you have any issues with that idea.

@owahltinez Thanks for the stackoverflow reference.
Do you want accreditation as the data source / compiler on my page?
If so what would you like? Name, link etc.

There is no need for citation / source, but if you want to add one you can say: Open COVID-19 Dataset and link to https://github.com/open-covid-19/data.
If you care to share a link to your page, I can add it to the README file in this repo.

Ok, thanks;
Link to site : http://www.starlords3k.com/covid19.php

@Mahks I'm completely blown away with your site, it is very cool!
I added a call-out to the README, please take a look and let me know if you are not happy with it.

Thanks for the call-out, glad you like it.
I am looking to add option to display dates of major events, like when a lock down began. Do you know of any data resource for that?
I would also like to find a data source for country statistics like level of medical care, doctors per population etc.

I would also like to find a data source for country statistics like level of medical care, doctors per population etc.

That was discussed briefly in this other issue. I think the best source of information for that should be Wikipedia, but the information is quite sparse. A few examples:

List of hospital beds by country
List of physicians relative to population size by country

I don't think we should add too many columns of metadata to the output data CSV file here, but I wouldn't be opposed to adding the columns to the metadata CSV file as long as they are not expected to change often (like number of hospital beds or physicians).
For something that is expected to change often and has a date associated with it, like government measures or construction of field hospitals, I would love to find a reliable source of information that we can automatically scrape and add to the output data CSV file. Ideally that would be on Wikipedia, which is both easy to scrape and accessible to everyone.

Please review my lack of results.
https://kepler.gl/demo/map?mapUrl=https://dl.dropboxusercontent.com/s/lrb24g5cc1c15ja/COVID-19_Dataset.json

Sorry I did not mean to suggest you add that other data to your file.
I wanted it for my own use.
There is this api : https://apps.who.int/gho/athena/public_docs/examples.html#ex2
Lots of data, but does not seem to be real current.
I wondered why you had all the country data repeated in your csv file.
Why not have 2 files, one for country, one for data per date?
For my site I parse your file into JSON format : http://www.starlords3k.com/covid19_data.json
Was only 200Kb until I added the worldwide numbers.

Please review my poor-quality work

@quixote79 I think that looks great! I don't think I've seen any other implementations using MapBox, everyone else appears to be using ArcGIS. Do you want a call-out in the README of this repo using that link?

Sorry I did not mean to suggest you add that other data to your file.
I wanted it for my own use.

@Mahks well, I think it would be useful to everyone else. My goal for this repo is to serve as a source of well-maintained, high-quality data related to COVID-19.

There is this api : https://apps.who.int/gho/athena/public_docs/examples.html#ex2
Lots of data, but does not seem to be real current.

Thanks for the link. I'll take a look but I'm not very hopeful since I've been disappointed with the quality of the data coming from WHO.

I wondered why you had all the country data repeated in your csv file.
Why not have 2 files, one for country, one for data per date?
For my site I parse your file into JSON format : http://www.starlords3k.com/covid19_data.json
Was only 200Kb until I added the worldwide numbers.

Yes, the current solution is not optimal. I think a lot of people are building maps and the latitude/longitude + population did not make the final output unreasonably large but I would not want to add any more fields. The main issue is that performing joins with non-tabulated formats like JSON is a pain, and some of the map tools people are using simply can't do that: you provide a JSON file and then use a GUI to create a map.
What do you think about this idea:

Create data.min.csv and data.min.json versions of the dataset with only Key, Confirmed and Deaths columns, where Key is CountryCode for country-level data and ${CountryCode}_${RegionCode} for region-level data.
Add the Key column to metadata.csv and publish that and a metadata.json in the output folder, and over time add more columns for things that we can get information for a reasonable number of countries.

It still would not be like the JSON file you linked, because you are pivoting the data away from the record format and using country codes as keys. I would not recommend creating your own JSON files since it will add yet another hop with additional delay -- there are currently 1-2 hops between the data coming from an authoritative source until it gets published in this repo, depending on the data source. I don't yet have it fully automated because some sources have not been 100% stable, which means that there could be multiple hours of delay right now. Once I can trust the data sources a little bit more, I hope to automatically update the data hourly.

I don't think you have to provide JSON format, as anyone needing that could easily convert from csv. The JSON file above was created just to demonstrate the structure.
I use your csv file and convert with PHP into JSON to send to the browser. So there is no "another hop with additional delay".
What I really meant I guess was, why not have 2 csv files? One with the country data and another with the date data. I don't know if that would be problematic for other uses. Does the data have to be in one file for some reason?
country_data.csv : AF,Afghanistan,,,33.93911,67.709953,38041754
ie; code,name,code,name,lat,long,pop
date_data.csv: 2019-12-31,1,0
ie; date,cases,dead
It does not matter for me as I can parse the data regardless. Mostly wonder why the repetition of data. file is 400% larger as a result.

@quixote79 I think that looks great! I don't think I've seen any other implementations using MapBox, everyone else appears to be using ArcGIS. Do you want a call-out in the README of this repo using that link?

Yes, please :)

@quixote79 I added a call-out to your map in the README, please take a look!

@Mahks

I don't think you have to provide JSON format, as anyone needing that could easily convert from csv.

I don't think it's strictly a need, but the cost to do it is very low and reading JSON is much easier than reading CSV from Javascript applications that have no server-side component.

I use your csv file and convert with PHP into JSON to send to the browser. So there is no "another hop with additional delay".

Ah, I see. I thought you were converting to JSON on a schedule. That's an interesting architecture, and I'm guessing that you are caching the resulting JSON file (which is why you were asking about last updated time).

What I really meant I guess was, why not have 2 csv files? One with the country data and another with the date data. I don't know if that would be problematic for other uses. Does the data have to be in one file for some reason?

I will provide what you are asking for in the form of data_minimal.csv and data_minimal.json, I think it's a good idea for those that care about performance; even though data.json is currently 2.6MB, which is manageable, it's only going to get bigger. data.csv is still under 1MB which is much better, but as you point out it's simply not very efficient.
Unfortunately, having the latitude/longitude and population data is necessary for those who want to use a map tool that only accepts a single data source, and may not be very skilled programmers with their own server to do re-processing of the CSV / JSON files provided here. By providing both data.csv and data_minimal.csv we should be able to make everyone happy without incurring too much of a maintenance burden.

@Mahks please take a look at the new metadata and data_minimal CSV files, I think they should correspond to what you were suggesting. To join the datasets, simply use the column Key -- use an outer join since there will be multiple keys in the data_minimal dataset, one for each date where data is available.

I think your link to data_minimal.csv is pointing to the wrong file...
metadata.csv looks great
On Wed, 1 Apr 2020 at 02:48, Oscar Wahltinez<EMAIL_ADDRESS>wrote:

@Mahks https://github.com/Mahks please take a look at the new metadata
https://open-covid-19.github.io/data/metadata.csv and data_minimal
https://open-covid-19.github.io/data/data_minimal.csv CSV files, I
think they should correspond to what you were suggesting. To join the
datasets, simply use the column Key -- use an outer join since there will
be multiple keys in the data_minimal dataset, one for each date where data
is available.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-covid-19/data/issues/23#issuecomment-606837141,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ5P2IUH7Y4T36C2DIUVVTRKJCH5ANCNFSM4LUDGQMA
.

You changed the current data format!!!
It killed my site :(
On Wed, 1 Apr 2020 at 05:32, Mahks Doma<EMAIL_ADDRESS>wrote:

I think your link to data_minimal.csv is pointing to the wrong file...
metadata.csv looks great
On Wed, 1 Apr 2020 at 02:48, Oscar Wahltinez<EMAIL_ADDRESS>wrote:

@Mahks https://github.com/Mahks please take a look at the new metadata
https://open-covid-19.github.io/data/metadata.csv and data_minimal
https://open-covid-19.github.io/data/data_minimal.csv CSV files, I
think they should correspond to what you were suggesting. To join the
datasets, simply use the column Key -- use an outer join since there will
be multiple keys in the data_minimal dataset, one for each date where data
is available.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-covid-19/data/issues/23#issuecomment-606837141,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ5P2IUH7Y4T36C2DIUVVTRKJCH5ANCNFSM4LUDGQMA
.

Uh oh. What are you seeing differently? I'm looking into it right now

https://raw.githubusercontent.com/open-covid-19/data/master/output/data.csv
Arrghh
On Wed, 1 Apr 2020 at 05:51, Oscar Wahltinez<EMAIL_ADDRESS>wrote:

Uh oh. What are you seeing differently? I'm looking into it right now
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-covid-19/data/issues/23#issuecomment-606925855,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ5P2ILOXCICHLO5FWWXT3RKJXVPANCNFSM4LUDGQMA
.

That is the file I use to get the data. I can't access the other with PHP
(so far as I have found)
On Wed, 1 Apr 2020 at 06:11, Mahks Doma<EMAIL_ADDRESS>wrote:

https://raw.githubusercontent.com/open-covid-19/data/master/output/data.csv
Arrghh
On Wed, 1 Apr 2020 at 05:51, Oscar Wahltinez<EMAIL_ADDRESS>wrote:

Uh oh. What are you seeing differently? I'm looking into it right now
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-covid-19/data/issues/23#issuecomment-606925855,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ5P2ILOXCICHLO5FWWXT3RKJXVPANCNFSM4LUDGQMA
.

OK, I understand the problem. You are using the raw github output. Please use instead this link: https://open-covid-19.github.io/data/data.csv

do you use discord? https://discord.gg/TxUkaqy
On Wed, 1 Apr 2020 at 06:12, Mahks Doma<EMAIL_ADDRESS>wrote:

That is the file I use to get the data. I can't access the other with PHP
(so far as I have found)
On Wed, 1 Apr 2020 at 06:11, Mahks Doma<EMAIL_ADDRESS>wrote:

https://raw.githubusercontent.com/open-covid-19/data/master/output/data.csv
Arrghh
On Wed, 1 Apr 2020 at 05:51, Oscar Wahltinez<EMAIL_ADDRESS>wrote:

Uh oh. What are you seeing differently? I'm looking into it right now
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-covid-19/data/issues/23#issuecomment-606925855,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQ5P2ILOXCICHLO5FWWXT3RKJXVPANCNFSM4LUDGQMA
.

Closing this issue, since all the problems listed here have been resolved. Feel free to open new issues to provide feedback or ask questions!


Make the Link Checker API absent in Carrenza Prod and Staging

The Link Checker API is now running in AWS, but still running (but
should be unused) in Carrenza Production and Carrenza Staging.
This old, unused version of the app still runinng in Carrenza is
potentially confusing, and consumes some resources. Therefore, this
commit makes it absent.

runinng typo in commit. Code looks good.

Good spot, I've fixed the typo.


The return of Issue #6: Cannot read property 'displayName' of undefined

Hola !
I encountered a problem that seems very similar to https://github.com/bruderstein/unexpected-react/issues/6
I've been trying for the whole afternoon to figure out a way to reproduce it with a simplified example and I finaaaaaallyyyy found what cause the problem(I can now continue my day in peace).
First, here's the error:

Uncaught TypeError: Cannot read property 'displayName' of undefined
at ReactElementAdapter.getName (node_modules/unexpected-htmllike-jsx-adapter/lib/ReactElementAdapter.js:79:32)
at convertToDiff (node_modules/unexpected-htmllike/lib/convertToDiff.js:29:23)
at node_modules/unexpected-htmllike/lib/diff.js:538:69
at Array.forEach (native)
at node_modules/unexpected-htmllike/lib/diff.js:531:21
at node_modules/array-changes-async/lib/arrayChanges.js:224:17
at setEqual (node_modules/array-changes-async/lib/arrayChanges.js:199:28)
at node_modules/array-changes-async/lib/arrayChanges.js:212:25
at node_modules/unexpected-htmllike/lib/diff.js:505:24
at tryCatcher (node_modules/unexpected/node_modules/bluebird/js/main/util.js:26:23)
at Promise._settlePromiseFromHandler (node_modules/unexpected/node_modules/bluebird/js/main/promise.js:503:31)
at Promise._settlePromiseAt (node_modules/unexpected/node_modules/bluebird/js/main/promise.js:577:18)
at Async._drainQueue (node_modules/unexpected/node_modules/bluebird/js/main/async.js:128:12)
at Async._drainQueues (node_modules/unexpected/node_modules/bluebird/js/main/async.js:133:10)
at Async.drainQueues (node_modules/unexpected/node_modules/bluebird/js/main/async.js:15:14)
at node_modules/unexpected/lib/workQueue.js:8:13
at Array.forEach (native)
at Object.workQueue.drain (node_modules/unexpected/lib/workQueue.js:7:20)
at Immediate._onImmediate (node_modules/unexpected/lib/workQueue.js:17:19)

Here's how to reproduce it (I copy-pasted the required setup fromthe doc):
class LiElement extends React.Component {
  render () {
    return
{this.props.name}

  }
}

LiElement.propTypes = {
  name: React.PropTypes.string
}

class UlElement extends React.Component {
  render () {
    // if I change the 'LiElement' for a 'li' directly, no problem
    return (

        { this.props.items.map(i => <LiElement key={i.id} name={i.name} />) }

    )
  }
}

UlElement.propTypes = {
  items: React.PropTypes.array
}

const ReactTestUtils = require('react-addons-test-utils')

describe('Test', function () {
  const items = [
    { id: 0, name: 'Banana' },
    { id: 1, name: 'Chocolate' },
    { id: 2, name: 'Mustard' }
  ]

  it('should render liElement', function () {
    let component = ReactTestUtils.renderIntoDocument(
      <UlElement items={items} />
    )

    return expect(component, 'to have rendered with all children', (

{items[0].name}

{items[2].name}

{items[1].name}

    )) // displayName if not in the right order, no error otherwise
  })
})

Ok, I can reproduce it - thanks for a great bug report.
I'm investigating...

Wow. This a great bug. I've found the issue. I should have it fixed today or latest tomorrow (the fix is pretty simple, but I want to add some tests around it).

@yormi I've just published 0.5.2, which forces the update of unexpected-htmllike, which fixes this issue.  Thanks again for detailed bug report - I've added it in as a regression test.

Awesome man ! You are a really quick problem solver :) These days, I'm in holidays but I'll upgrade when I come back. Thank you for your great work ! :D

Thank you very much. Also met the bug.

I also ran into this bug when compiling with the transform-react-constant-elements plugin from babel-react-optimize. I was able to workaround the issue by using the plugins explicitly (and omitting transform-react-constant-elements rather than using the Babel preset itself.
It'd be great if unexpected-html could not blow up when displayName is not available as I'd like to be able to test my components as they appear in production, but I realize this may not be a the common path and am OK moving forward with my workaround. Wanted to share my experience in case anyone else has the same setup as I do.

It shouldn't blow up, you're right. I'll take a look - there must be some form of name / type there, otherwise React wouldn't be able to reconcile. It'll probably be a week or so before I can look at this though.


Hook for a matplotlib function

While using the bokeh backend to plot a Curve element, I would like to use the fill_between function (matplotlib.pyplot.fill_between) that matplotlib has to color the area between two curves.
I thought I could make it possible by using hooks, but I don't know the proper plot.handle to do that. The following picture illustrates what I meant.

You cannot use natively use matplotlib functions in the bokeh backend, the two are entirely different worlds. You have two choices:

Either use the matplotlib backend, and use a hook as described at http://holoviews.org/user_guide/Customizing_Plots.html#Plot-hooks
You should also be able to achieve the same without hooks, using http://holoviews.org/reference/elements/bokeh/Area.html

Also for the future, consider posting usage questions like these on stackoverflow :)

Right, (a) SO is better for such questions, and (b) you should be able to easily get a plot like that by overlaying a thresholded version of your data plotted as hv.Area on top of one plotted as hv.Curve.


net/vnstat: make plugin compatible to new v2 version

Removed weekly, added a note for wiping DB in plugin changelog

Merged, thanks!


Feature request: Upgrate cvxpy to 1.1.1

Upgrade cvxpy to 1.1.1
When upgrading, the method https://github.com/robertmartin8/PyPortfolioOpt/blob/master/pypfopt/discrete_allocation.py#L244 will fail

I'm still trying to figure out what to do about that. They've removed the ECOS_BB default solver, so users would have to explicitly install a mixed-integer solver. Will think about it some more

No problem. I installed http://cvxopt.org/ and run the code. It is not breaking but the method is returning wrong values.
Once you have an idea, if you wish, I can run the tests that I have in my project and provide some feedback.
Thanks!

When you say incorrect, do you mean that the resulting discrete allocations
aren't optimal? Or do they just not make sense
On Fri, 17 Jul 2020, 19:35 Diego Lovison<EMAIL_ADDRESS>wrote:

No problem. I installed http://cvxopt.org/ and ran the code. It is not
breaking but the method is returning wrong values.
Once you have an idea, if you wish, I can run the tests that I have in my
project and provide some feedback.
Thanks!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/robertmartin8/PyPortfolioOpt/issues/153#issuecomment-660058471,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACZ7ECLTIYIIHMU62S2VCSDR4AZPVANCNFSM4O4JUH4A
.

They don't make sense.

Fixed!


Throw meaningful error on org catalog fail

Dealing with failing remote servers 'properly' requires retry, crons and other things best not done in a hurry. In the meantime, I've added meaningful error messages which would have saved us time this week, eg
Unable to set Peer Org DGS: ApolloError: fetch failed
To test this I changed the peerUrl of DGS to https://diegesis.biblex/graphql.

Could you please provide me with the needed changes in the config?
I need to add DGS to 'orgs', but there is also a stanza that I need to add to 'orgsConfig'.


About training speed.

Thanks for the great work! I wonder about the training speed? I tried to train the model on 2*V100 with batch size 256, but the training process is too slow and about 10 hours per epoch, is it normal?

I have fixed the problem, and it takes about 1000s for training and 8min for validation per epoch with batch size 32 on 2*V100, but I can only config the batch size as 16 per V100 with 32G memory or 8 for RTX2080 with 11G memory. If I try to enlarge the batch size, the oom error appears. It is abnormal and I will try to fix it.


Specialized Twitter Cards for each view

@xylodrone could help us on this page.
We can use the current template which is this one on Series

Let's use this as a template and add text under it.

Series should have "Series Insights"

Title: Series Insights
Description: Ape into NFTs based on data points

Spotlight -- "Creators Directory"

NFT Artist Rank
Discover new artists based on rank

Collections -- "Collections"

Collections
Collect new mints with collection insights

Gallery -- "Explore new mints"

Explorer
Ape into newest mints

Figma Files: https://www.figma.com/file/tcq6xYg9aq06c9CPwXbgSg/?node-id=30%3A2

Thinking how it would looks like once online as open graph cards are bit small, let's try with this one, better than current ones 😄

On twitter card validator seems distorted

Even on normal twitter somehow it's blurred?

but telegram looks good!

Thinking out loud.
@XyloDrone let's try put proportionally smaller (50%?) KodaDot logo, as branding and put that text bigger (+50%)?

Let me know your thoughts on this one. It's made in a smaller resolution:

What if we can make it even a bit bigger,
like edges will be horizontal aligned
Edges of border of text on the left, with logo on the right 👀

Update 3

:

Oh yeah, let's fire this #1149 @roiLeo 👀

Update 3
:

Here we go again | Resize images belo

w

Here we go again, resize below

Landing and Mint

Here we go again, resize below

pushed on #1149
What are the last two images for?

Yup 5th is for landing /
6th should go for /rmrk/mint /rmrk/create /permafrost/create

Reopen for #1274 as we don't have the v3 cards.
ping @XyloDrone

New cards here:
https://github.com/kodadot/weyland/issues/11

Lol, @vikiival accidentally close stuff, it's here @roiLeo https://github.com/kodadot/nft-gallery/issues/1529


Does this plugin support Android Studio 2.1 Preview 3?

I can not use it normally in Android Studio 2.1 Preview 3.

The plugin should be normally working even in previews.
Mind that when Android Studio updates it's usually necessary to reinstall the plugin.


RFC: SR-IOV Network Device Plugin - PCI address mismatch

Description of problem
Currently, the SR-IOV Network Device Plugin is the de-facto standard for managing SR-IOV-based network devices. When it is used to attach a device to a pod, it adds an ENV var to let the pod know how to access it. This ENV var has the following format:
PCIDEVICE_{RESOURCE_NAME} = pciAddress1[,pciAddress2,...]
This ENV is then used by a pod consuming this interface. A relevant example would be: a VFIO device being consumed by a DPDK application. Also, there are libraries that offer a programatic way of consuming this information: app-netutil
The problem when using the Device Plugin in Kata is that the PCI hierarchy is obviously different so the pod cannot properly consume the device it has been assigned.
Expected result
A mechanism should exist for the pod to access the guest's PCI address.
Ideally, the pod should see the same API. For that to happen Kata would need to have awareness of his naming convention and rewrite the address when it hotplugs the device
Another, less ideal, alternative would be to have the host->guest mapping exposed somehow and have the pod make the lookup / translation. This would make the pod aware of the fact its running on Kata, which is not good IMHO.
Opening this issue to get feedback from the community on this topic.
What do you think?

@amorenoz @bpradipt @dgibson can you take a look at this issue and clarify what still needs to be done?

Ping @dgibson

Assigning @amshinde as she is the main author of https://github.com/kata-containers/documentation/blob/master/use-cases/using-SRIOV-and-kata.md.

Actually I already have code for this, assigning myself.
A draft was in an earlier push of https://github.com/kata-containers/agent/pull/845, though it needs a bit of revision and I'm going to pull it out of that issue into another.


load more is not working

Hi, i’m trying your code example for the load more button but it doesn’t work in my website. It doesn’t load any more pic. I have set the limit to 60 and using this
$('#load-more').on('click', function() {
feed.next();
});
Any help? thank you

You need to define .next.
Use this working code example: http://jsfiddle.net/unuLyzx0/122/


Need an instruction on self-host implementation

I'd like to implement those GitHub icons on my web site without linking to external resource files. I mean instead of
<script async defer src="https://buttons.github.io/buttons.js"></script>

, I'd like to have
<script async defer src="https://{my-site}/buttons.js"></script>

It seems I've succeeded doing it. But since there is no instruction for it, I'm not sure if I'm missing something or not. What I did was below.

Copied these files into the my site asset directory.

- asset/github-buttons/buttons.html
- asset/github-buttons/buttons.js
- asset/github-buttons/assets/css/buttons.css
- asset/github-buttons/assets/css/main.css
- asset/github-buttons/assets/js/app.min.js
- asset/vue/vue.min.js

In HTML, added these in the <head> tag.

<link rel="stylesheet" href="asset/github-buttons/assets/css/main.css">
<script src="asset/vue/vue.min.js"></script><!-- for github-buttons -->

and this at the bottom.
<script async defer src="asset/github-buttons/assets/js/app.min.js"></script>

Is this the right way of doing it? Just in case if I'm missing something.

As I already commented in the other issue, if you don't need the UI to generate buttons, you only need buttons.js (or minified buttons.min.js) and buttons.html (for legacy browser support). The two file must be served in the same directory and, you will also need to change one hard coded URL (again, for legacy browsers) in buttons.js.
The goal of this project is to provide a buttons as service. Self-hosting is permitted by LICENSE, but you do so at your own risk.


We default allow_access_to_external_requester_pays_buckets to true - should we?

I was surprised to see that we have terraform variable templates defaulting allow_access_to_external_requester_pays_buckets to true. Should we really default this to true?
Anyone in a community could incur a major cloud cost by downloading massive amounts of data like this, so it would feel better if we make the community representative request its enabled by having it enabled by default - otherwise the options in my mind is that we must be able to communicate this better for any new hub community champion.

I see it default to true in the daskhub template https://github.com/2i2c-org/infrastructure/blob/467f197ca6fe1007550a99074ad9c6a71d0eafea/terraform/gcp/projects/daskhub-template.tfvars#L51
And not specified at all the basehub template.
And in the variable definitions file it deafults to false
https://github.com/2i2c-org/infrastructure/blob/467f197ca6fe1007550a99074ad9c6a71d0eafea/terraform/gcp/variables.tf#L405
I agree this feature should be an opt-in one (this is how we document it to be) and I believe we should remove the default altogether from the daskhub hub and let the default come from the variables file.
I think the reason for the current daskhub default is rooted into the confusion that was around what this config did and the lack of docs. So good thing we fixed that and you spotted this remaining issue 🚀

+1 on having it default to false.


Project Name and Mailing List

[ ] Assign a project name.  (Current name is Fast Numpy Loops) @jack-pappas @tdimitri any ideas?

[ ] Identify the community communication channels (Gitter, Mailing List, etc...) @jack-pappas @tdimitri any preferences here?

Sample name suggestions for to start the discussion:

numpy-threaded-hyperloops
numpy-accelerated-loops
numpy-threading-extensions (existing repo name)

Closing, we have pnumpy and a Google Groups mailing list


WebSocket permessage-deflate integration

Defaults to off, so rippled only blows up if you tell it to

Beast is missing the code that applies the compression settings to the zlib compressor! Back to the drawing board...

False alarm, investigating...

Still 👍

Merged as https://github.com/ripple/rippled/pull/2005/commits/ce7e83f7639148032b5ba6ac1ba20dd6e2b8ea43 and https://github.com/ripple/rippled/pull/2005/commits/f6a0345831242648d9aa0957e3d252a9e4304bdb.


Turn off datepicker validation

I do not need to validate value on every keypress.
Angular: 1.5.5
UIBS: datepicker
Bootstrap: 3

@daviatorstorm to be quite honest, as we are all unpaid volunteers, I don't think we are going to be able to commit resources to doing this.  if you're finding that your app is really sluggish because of this then you might want to start with one of the Angular watchers plugins which will show you how many watches you have on a given page.
I'm going to close this as won't support, but please feel free to convince me otherwise.  Thanks.

I need to validate by untouch with 'dd.MM.yyyy', 'd.MM.yyyy', 'dd.M.yyyy', 'd.M.yyyy'.
How can I do it with current validation?

I found a solution for that :)
Just pasted right parameters to alt-input-formats and create a attribute restricted directive with validation on input blur.
altFormats: 'd!', 'd!.', 'd!.M!', 'd!.M!.', 'd!.M!.y', 'd!.M!.y'.


Allow clients to assign metadata to queries

My Environment

ArangoDB Version:        3.4.2
Storage Engine:             RocksDB
Deployment Mode:       Single Server
Deployment Strategy:    ArangoDB Starter in Docker
Infrastructure:               own
Operating System:        MacOS 10.13.4
Total RAM in your machine:        32Gb
Disks in use:        SSD
Used Package:              Docker - official Docker library

This is more of a suggestion/feature request. Why not allow clients to assign metadata to queries, for example, using key/values pairs on AQL Options. This would allow clients to quickly identify queries that they fired to the server and act on them (for example, long-running queries)
This information would then be available in /_api/query/current

Hi,
It is actually implemented in the server ( see https://docs.arangodb.com/3.4/HTTP/AsyncResultsManagement/ for more details ) - However, not many drivers implement the client API for this.

From what I could find out from talking on Slack, this is what the Web UI is doing.
I had a quick look and saw the results are actually coming from the Job API.
Another thing I was told is that I could combine the AQL streaming cursors with the Job API.
Currently using the Official Arango Java Driver and doubt it supports any of this.

the java driver was actually the first to implement this.

Since the referenced ArangoDB 3.4 is EOLed closing this as solved.
Please note that ArangoDB 3.7 is available.


Fix C++11 support in CMake and headers.

It seems that compiling software that uses C++11 against Siconos would successfully compile, but not link.  It turned out be because, by default, Siconos compiles without -std=c++11, and uses boost::shared_ptr, etc.  However, the use of boost::shared_ptr or std::shared_ptr is detected by the preprocessor by checking against __cplusplus in some headers.  So software including those headers when -std=c++11 was enabled would incorrectly choose the std:: namespace, when Siconos was compiled with the boost:: namespace.
This patch series:

Adds USE_CXX11 to CMake, which requests -std=c++11 enabled, and checks that it works during configuration.  An error is emitted if it is requested but not supported.  It is disabled by default.
Fixes some problematic areas in mechanics where boost::array or similar was used even when c++11 was enabled.
Replaces checks against __cplusplus in the headers with checks against SICONOS_CXXVERSION, which is hardcoded at configure time, guaranteeing that user software will select the shared_ptr type that Siconos was compiled with.

Good idea!

Okay I have been looking a little deeper into CMake support for C++11, and here's a slightly different proposal:

I tried to "do the right thing" and use CMAKE_CXX_STANDARD and module WriteCompilerDetectionHeader along with target_compile_features() for detecting and specifying specific C++ features.  CMake is then responsible for determining the C++ standard to use and how to use it.
However, I discovered that mysteriously CMake doesn't provide any feature named "cxx_memory_shared_ptr" or similar -- so it's a bit of a dead-end to let CMake take care of this.

However, the idea of detecting specific features is good.  So, I propose:

User can set CMAKE_CXX_STANDARD and USE_BOOST_FOR_CXX11 to whatever he wants.

Therefore we presume he knows that enabling CMAKE_CXX_STANDARD=11 will enable namespace SP=std, unless USE_BOOST_FOR_CXX11 is enabled.

If USE_BOOST_FOR_CXX11 not set, check if std::shared_ptr is supported.

If yes, use namespace SP=std and set a proprocessor variable SICONOS_STD_SHARED_PTR
Otherwise, do not use it.

We mostly forget about SICONOS_CXXVERSION, and use feature variables in all cases.
Variables SICONOS_STD_SHARED_PTR and SICONOS_USE_BOOST_FOR_CXX11 are recorded to SiconosConfig.h and used in headers


Examine framework shutdown on interrupt (Ctrl-C)

When the framework scheduler is interrupted, mesos shuts down that framework's executors as well.


Adds validation for the action groups type key

Description
Added validation for the action groups type key.
Issues Resolved
#4374
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

The change looks good to me @willyborankin. Seems like we could a couple more test cases according to code-cov?

Sure I will extend it together with https://github.com/opensearch-project/security/issues/4393 and a new test for roles.

And I checked the report it complains on the variables we test already in the integration test:
https://app.codecov.io/gh/opensearch-project/security/pull/4400?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=opensearch-project

Changes look good to me @willyborankin. Can you check the code-cov annotations? I think we should add a test that covers the deprecation message.

Well it is a logging message not sure that we need to test it. But the first one I fixed.


Crash in get_client_addr()

After working through issues #1401 and #1443, we now see the following crash in our copy of 7.1.  The crash occurs once every couple hours.
#0  0x00002b46debb3867 in ?? () from /lib64/libgcc_s.so.1
#1  0x00002b46debb4119 in _Unwind_Backtrace () from /lib64/libgcc_s.so.1
#2  0x00002b46deeb9936 in backtrace () from /lib64/libc.so.6
#3  0x00002b46dc70fe72 in ink_stack_trace_dump () at ink_stack_trace.cc:61
#4  0x00002b46dc71213a in signal_crash_handler (signo=11) at signals.cc:186
#5  0x0000000000513414 in crash_logger_invoke (signo=11, info=0x2b46e39dc130, ctx=0x2b46e39dc000) at Crash.cc:169
#6  <signal handler called>
#7  0x0000000000000001 in ?? () at ../../lib/ts/Diags.h:142
#8  0x00000000005126d5 in NetVConnection::get_remote_addr (this=0x2aad1cc4d0a0)
    at /home/shinrich/yats_build/trafficserver/iocore/net/P_NetVConnection.h:30
#9  0x000000000055deea in ProxyClientSession::get_client_addr (this=0x2ab0dc25bba0) at ProxyClientSession.h:221
#10 0x00000000005352bf in TSHttpSsnClientAddrGet (ssnp=0x2ab0dc25bba0) at InkAPI.cc:5439
#11 0x000000000053530d in TSHttpTxnClientAddrGet (txnp=0x2b474440c9f0) at InkAPI.cc:5447
#12 0x00002aaaae4008f6 in http_hook (contp=0x1e42f40, event=60006, edata=0x2b474440c9f0) at INKPluginInit.cc:174
#13 0x000000000052a5fd in INKContInternal::handle_event (this=0x1e42f40, event=60006, edata=0x2b474440c9f0) at InkAPI.cc:1048
#14 0x00000000005160f2 in Continuation::handleEvent (this=0x1e42f40, event=60006, data=0x2b474440c9f0)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#15 0x000000000052adf3 in APIHook::invoke (this=0x1e44f40, event=60006, edata=0x2b474440c9f0) at InkAPI.cc:1267
#16 0x00000000005e5e16 in HttpSM::state_api_callout (this=0x2b474440c9f0, event=0, data=0x0) at HttpSM.cc:1462
#17 0x00000000005f3a11 in HttpSM::do_api_callout_internal (this=0x2b474440c9f0) at HttpSM.cc:5171
#18 0x0000000000601b85 in HttpSM::do_api_callout (this=0x2b474440c9f0) at HttpSM.cc:438
#19 0x00000000005e78d6 in HttpSM::state_read_server_response_header (this=0x2b474440c9f0, event=100, data=0x2b4744cac508)
    at HttpSM.cc:1962
#20 0x00000000005ea331 in HttpSM::main_handler (this=0x2b474440c9f0, event=100, data=0x2b4744cac508) at HttpSM.cc:2662
#21 0x00000000005160f2 in Continuation::handleEvent (this=0x2b474440c9f0, event=100, data=0x2b4744cac508)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#22 0x000000000078489f in read_signal_and_update (event=100, vc=0x2b4744cac3e0) at UnixNetVConnection.cc:145
#23 0x0000000000787a10 in UnixNetVConnection::readSignalAndUpdate (this=0x2b4744cac3e0, event=100) at UnixNetVConnection.cc:1125
#24 0x000000000076a098 in SSLNetVConnection::net_read_io (this=0x2b4744cac3e0, nh=0x2b46e18ade60, lthread=0x2b46e18aa010)
    at SSLNetVConnection.cc:587
#25 0x000000000077d20d in NetHandler::mainNetEvent (this=0x2b46e18ade60, event=5, e=0x1873f20) at UnixNet.cc:509
#26 0x00000000005160f2 in Continuation::handleEvent (this=0x2b46e18ade60, event=5, data=0x1873f20)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#27 0x00000000007a69c5 in EThread::process_event (this=0x2b46e18aa010, e=0x1873f20, calling_code=5) at UnixEThread.cc:143
#28 0x00000000007a6eb1 in EThread::execute (this=0x2b46e18aa010) at UnixEThread.cc:270
#29 0x00000000007a6089 in spawn_thread_internal (a=0x1762fe0) at Thread.cc:84
#30 0x00002b46df156aa1 in start_thread () from /lib64/libpthread.so.0
#31 0x00002b46deea393d in clone () from /lib64/libc.so.6

I assume that there has been a goodly amount of inlining here so it is hard to tell how we got from NetVConnection::get_remote_addr to the Diags::on method.  Digging through the core, the cached member addresses seem ok, and I don't see evidence of thread races.
I think I have seen this stack reported before, but couldn't find the issue.

Very odd bug.  Most of the crashes are in get_client_addr or get_remote_addr.  In both cases, the problem is at the point of the virtual method call into the corresponding test.  The RAX register is either 0 or 1.  But when you go through the assembly instructions based on the current register values, the function pointer should be valid.
Tried this on two different machines in case it was a HW problem.  Still see the problem on both machine.   Set counters to see if the crash happens when calling into the set method or on return.  The crash happens before the set method is called.
At this point, I think it is a use-after-free problem.  If I do the function calculation off a VC object (which would be the pointer in the virtual function table after free), the value is a 0 or 1.
I've tried to set up ASAN builds but they are not working for me at the moment (issue filed).

Still struggling with this issue.  Running into it every 2-4 hours in production.  At this point, I think it is a use-after-free, but as noted avoid ASAN fails in this case.
Looking at a core more closely, it seems to be an incoming client VC that is being messed up.  In the core, the remote_addr of the VC did not match the src_addr in the HttpSM.  Both IP addresses show up in our squid.log at roughly the same time.
We are running with accept threads.  I'm going to turn that off on my test machine,  I assume that will eliminate this race condition.  If so, then will need to press on and see how we are freeing the VC early.

Have been adding member variables to NetVConnection and adding assert to HttpSM::state_api_callout.
In the cores it seemed that the crash always happened in the first hook called from HttpSM::state_read_server_response_header.  As noted in the previous comment, the event came in on the server side VC, and the state_read_server_response_header is getting ready to send the response of the UA side VC.   I added allocate thread, free_thread, and last_alloc time to the netvc.  And added the following assert at the beginning of NetVConnection
{code}
ink_release_assert(milestones[TS_MILESTONE_UA_CLOSE] > 0 || ua_session->get_netvc() == NULL ||
(ua_session->get_create_time() != 0 && ink_hrtime_to_msec(ua_session->get_create_time()) <= ink_hrtime_to_msec(milestones[TS_MILESTONE_UA_BEGIN])));
{code}
This will trigger if there is not last_alloc_time set (it has been freed) or the create time of the VC is newer than the start of the transaction (has been reallocated).  This assert would trigger before we called the hooks implying that the client_vc has been freed in a previous stack but the ua_session reference has not been cleaned up.  The only place I could see this happening is in read_signal_and_update and write_signal_and_update of the corresponding vio._cont is NULL.  I added warnings in the NULL path, and they triggered a handful of times with event=EVENT_ERROR (3).
The error bubble addition (which we were concerned about from issue #1401) adds calls to read_signal_error and write_signal_error.  So I reverted that commit (c1ac5f8bf87fd4bc3a8e06507219970d83965acd) and removed my workarounds for issue #1401.  I let it run overnight and it crashed twice on my assert, but in the accept stack not the send response stack.  The timestamps varied in the low microseconds, so I am doubting the accuracy of our timestamping at the micro and nano seconds.  I added the ink_hrtime_to_msec to the asserts, and kicked it off to run today.

This is an automatically generated message.
<EMAIL_ADDRESS>is no longer with Yahoo! Inc.
Your message will not be forwarded.
If you require assistance with a legal matter, please send a message to<EMAIL_ADDRESS>Thank you!

I had two more asserts once I added back in the accept thread.  Upon deeper inspection, the last alloc time and the UA_BEGIN time were off by half a ms, so I am attributing that to clock skew between threads.  I think the best action for 7.1.0 is to revert commit c1ac5f8bf87fd4bc3a8e06507219970d83965acd and investigate how to safely bubble up errors for 7.1.1.

I think this is resolved with commit 985a5d9e21f2938002295ea6af358df50c12873c on 7.1
Commit a128d56aee215ac5f74e449609317104ec3a76d7 on master.


Better function names?

Hey guys,
I do not know about you, but the functions names are not readable at all! Too cryptic for me.
What do you think Marco? Can we refactor them?
Thanks

@GeoDoo afaik the names are the ones suggested by Bartosz in his blogposts. Maybe it would be enough to provide some code comments referring each function to the challenge it belongs to.


Build Failure

Trying to build with julia 0.5 I got the following error:
jared@k475:~/.julia/v0.5/Clang/deps$ julia5 ./build.jl
g++ wrapclang.cpp -fPIC -c -o wrapclang.cpp.o -I/home/jared/build/clang_3.7/include -g -I/home/jared/build/clang_3.7/include  -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti  -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fPIC
g++: error: unrecognized command line option ‘-Wcovered-switch-default’
make: *** [wrapclang.cpp.o] Error 1
ERROR: LoadError: clang headers could not be found and BUILD_LLVM_CLANG=1 not specified in Make.user
Try setting BUILD_LLVM_CLANG=1 in Make.user in /mnt/external/build/julia_0.5/usr/bin and try again.
 in error(::String) at ./error.jl:21
 in include_from_node1(::String) at ./loading.jl:488
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /home/jared/.julia/v0.5/Clang/deps/build.jl, in expression starting on line 59

I am using g++ 4.8.4

I am no expert but I am going through the same process of building Clang.jl. Did you build Julia from source with BUILD_LLVM_CLANG=1 in Make.user? Which llvm are you using?

I did not add BUILD_LLVM_CLANG=1 in Make.user, but I did download clang+llvm-3.7.0-x86_64_linux-gnu-ubuntu-14.04.tar.xz and prepend its /bin to my PATH, its /lib to my LD_LIBRARY_PATH, and its /include to my CPATH.
Out of curiosity, when executing this line:
g++ wrapclang.cpp -fPIC -c -o wrapclang.cpp.o -I/home/jared/build/clang_3.7/include -g -I/home/jared/build/clang_3.7/include  -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG -fno-exceptions -fno-rtti  -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fPIC

did you use g++ or clang as the compiler?  I googled a bit and I don't think any versions of g++ support the option -Wcovered-switch-default.

I'm now getting a successful build with the latest commit, but when I go to load the package I get the following error.  Here are the results:
julia> Pkg.build("Clang")
INFO: Building Clang
g++ wrapclang.cpp.o -rdynamic -shared -o libwrapclang.so -L/usr/lib/llvm-3.9/lib  -L/usr/lib/llvm-3.9/lib -lclang

julia> using Clang.cindex
: CommandLine Error: Option 'help-list' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options

I am using ubuntu 16.10 with llvm/clang 3.9. This problem seems to be related to JuliaLang/julia#12644 which shows that other packages are having the same problem.


Fix enforcement on required=True on multi_choice

Also set default on many-to-many to required=False
Issue #201

Not sure if the default should be required=False for Field.multi_choice since this was effectively the behavior previously. Now I only changed the default factory setting for ManyToMany model fields.


Clarify Safari's support of MathML

Refs #2734; Safari 10+ fixes the issues referenced in the latter ticket.
The Mozilla MathML torture test doesn't have any issues in rendering common math symbols (e.g. summations, which were fairly buggy before). RTL support is pretty good (radicals render properly), and outstanding rendering issues with RTL math seem to be buggy rather than downright unsupported.

I suspect you meant this to change to a for partial support? p refers to "supported with polyfill".
Additionally you should add a numbered note to explain the partial support. Thanks!

My bad, I made a mental note not to get tripped up by it, but still did.
I've added number notes to the Safari data, and also added a direct link to the MathML torture test it references.

Looks great, thanks!


Update build workflow to run on PRs and manually

Signed-off-by: Shubham Sharma<EMAIL_ADDRESS>Description
Update the workflow file.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Closes #286
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation

Now the build step is also running!


Define a javascript module for JWTPublicKeyExtractor

Component to extract the json web key from the jwt token.

JWTPublicKeyExtractor

extractKey(oAuthToken Base64EncodedJWT) : JWK // Token from server
We assume the JWT contains a claim named "res_pub_key"
The format and the location of this information might be dependent on the target identity provider.

We prefer the IdP keeping this out of the access_token or refresh_token
IdP can put this in the oAuthToken
an identity provider might decide to put it accessToken because there is nothing else send to the client.
So the public key extractor will be configured to work with the target idp

This is obsolete. We use rfc7523 to retrieve resource server public key.


feat(tests): set --max-endpoints-per-slice=2 in K8s Kind clusters used in e2e tests

What this PR does / why we need it:
Since KIC should work with multiple EndpointSlices, it should be covered with tests too. By default 100 endpoints are allowed per EndpointSlice thus having Deployment with Service that uses multiple EndpointSlices is not feasible in CI tests.  This PR configures Kind clusters with --max-endpoints-per-slice 2 to make possible to test KIC against Services with multiple EndpointSlices.
Next PRs will focus to ensure that test cases actually test it.

Which issue this PR fixes:
This is a part of implementing e2e tests that verifies proper handling of EndpointSlices https://github.com/Kong/kubernetes-ingress-controller/issues/4026.

Special notes for your reviewer:
This is the part of implementing strategy proposed in this one too big PR https://github.com/Kong/kubernetes-ingress-controller/pull/4084.
Labelled with ci/run-e2e to be sure, result has to be examined.

PR Readiness Checklist:
Complete these before marking the PR as ready to review:

[ ] the CHANGELOG.md release notes have been updated to reflect any significant (and particularly user-facing) changes introduced by this PR

For ci/run-e2e on GKE image that does not include https://github.com/Kong/kubernetes-ingress-controller/pull/4071 (from yesterday was used, thus GKE results are ignored. For Kind everything is 🟢