Fixed #27226: don't add ETag and Last-Modified headers to the response in cache.patch_response_headers()

This is backwards-incompatible isn't it? Perhaps a deprecation similar to what's proposed in https://github.com/django/django/pull/6393 would solve the issue.

Do you propose to deprecate entire patch_response_headers()? And add another function instead?

https://github.com/django/django/pull/6393 deprecates the USE_ETAGS branch in patch_response_headers().
As for the 'Last-Modified' branch, it's not clear to me what problem it causes. Now these headers might disappear wherever the function is used? The ticket needs more explanation about the problems this solves and a consideration of backwards compatibility.

Ok with USE_ETAGS.
About Last-Modified - it is simply not needed. Nobody who uses patch_response_headers does check If-Modified-Since/If-Unmodified-Since from requests then.

I haven't really used these headers, but I don't understand what you're saying. How can you know which headers a client is looking at?

There is RFC7234 about how clients should treat HTTP-caching headers. Section 4.2.2 says:

A cache MUST NOT use heuristics to determine freshness when an explicit expiration time is present in the stored response.

In other words there is no reason to use Last-Modified among with valid Expires

Does the header cause some harm though?
Anyway, it might be safest to proceed with the deprecation in https://github.com/django/django/pull/6393 and then remove Last-Modified when that deprecation completes. Would you like to try updating that patch and addressing the comments?

I updated the USE_ETAGS deprecation patch in https://github.com/django/django/pull/7256. Feel free to review.
As for Last-Modified, it's been in Django since 1.0 (8aa98af6). I don't know if removing it will cause any harm or benefit so I guess I'm inclined to leave it unless you have a strong rationale for removing it. I can't think of a practical way to deprecate it, so I think we'd just have to remove it outright and call it a backwards-incompatible change.

There is one case where Last-Modified may lead to implicit caching of response by browsers (using special algorithm based on difference between date of Last-Modified and current date). But this works only if there is no any explicit HTTP-caching header in response: Cache-Control or Expires). And because patch_response_headers sets both of these headers there is no any benefit from Last-Modified. Also there is no harm if Last-Modified will be removed from patch_response_headers .
We can remove it now or we can remove it along with USE _ETAGS (#7256).

I didn't do a careful analysis, but could there be cases where Django makes use of Last-Modified such that if it's no longer set, could cause a behavior changed? e.g. https://github.com/django/django/blob/b5aac66b28c615b2bda63548cbd695dbb5a0c381/django/middleware/http.py#L20-L22

Clients use If-Modified-Since and If-Unmodified-Since in requests if there was Last-Modified header in previous answer from server. There are only two places where HTTP_IF_MODIFIED_SINCE/HTTP_IF_UNMODIFIED_SINCE are checked:

ConditionalGetMiddleware
last_modified decorator

And nothing in other place, including UpdateCacheMiddleware/FetchFromCacheMiddleware

I was thinking about a case where you have:
MIDDLWARE_CLASES = (
    'django.middleware.http.ConditionalGetMiddleware',
    'django.middleware.cache.UpdateCacheMiddleware',
    ...,
)

If patch_response_headers() (called by UpdateCacheMiddleware) no longer sets Last-Modified, then a last_modified value won't be passed to ConditionalGetMiddleware'sget_conditional_response(). I'm not sure if that could cause a problem or behavior difference in practice.
A found a few projects using GitHub code search that have the middleware ordering above although more often the order is swapped in which case I don't think there would be a behavior change.

But there is no way when Last-Modified generated from UpdateCacheMiddleware will take effect in ConditionalGetMiddleware in such case. Because UpdateCacheMiddleware always generates fresh one and ConditionalGetMiddleware does always miss (If-Modified-Since always less then Last-Modified).

Okay, can you update patch to remove only Last-Modified and also add a mention and explanation of the reason for the removal in the 1.11 release notes under "Backwards-incompatible changes"?

ok, I'll fix it

ready to check

merged in 9e07a9b5feb30fe06fb23d0e77edd34f6c3f92b5, thanks!


Bug: applyChanges / setAtPath - Error: Cannot set a value directly

TypeError: 'set' on proxy: trap returned falsish for property '***'
    at setAtPath (helpers.ts:85:16)
    at applyChange (helpers.ts:224:12)
    at applyChanges (helpers.ts:228:17)
    at channel.onmessage (state.ts:35:3)

https://github.com/LegendApp/legend-state/blob/3a7ba8c06c973fa4fd8d72a521c68a0fcb1a246a/src/helpers.ts#L85

applyChanges is not for observable. Didn't saw the typings definition, sorry.


picocrypt: 1.38 -> 1.40

Description of changes
https://github.com/Picocrypt/Picocrypt/releases/tag/1.39
https://github.com/Picocrypt/Picocrypt/releases/tag/1.40
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/4398

@NixOS/nixpkgs-merge-bot merge

the mergebot works only for automatic updates (created by r-ryantm) for now


costum_label

Hi deepof
I am trying to set up deepof for a multi-animal experiment. I am struggling with the body-part labeling: although I am using the deepof_8 scheme, my body-parts are named differently. Therefore, I created a custom label but I get the following error when I create the project:
`It seems you're using a custom labeling scheme which is missing key body parts. You can proceed, but not all areas will be computed.```
This is my custom labeling code :
deepof_custom = {
                "nose": ["leftear", "rightear"],
                "midbody": ["leftear", "rightear", "leftside", "rightside", "tailbase"],
                "tailbase": ["tailtip"],
            }

and this is my project file:

my_project = deepof.data.Project(
    project_path=r"C:\Users\odejul00\deepof",
    video_path=r"C:\Users\odejul00\deepof\Social_Videos",
    table_path=r"C:\Users\odejul00\deepof\tables",
    project_name="my_project",
video_scale=420,
arena="polygonal-manual",
animal_ids=["individual1", "individual2"],
    bodypart_graph=deepof_custom,
)

Also, when I try to run the unsupervised analysis, I get this error:
>>> supervised_annot = my_project.supervised_annotation()
Traceback (most recent call last):
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\indexes\base.py", line 3802, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas\_libs\index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas\_libs\hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'individual1_Center'

Is my custom labeling wrong or do you know why it doesn't apply the defined bodyparts?
Thank you :)

Hi Patrick
Thank you for your fast response. After creating the project:
my_project = my_project.create(verbose=True)
I run the following commands:
my_project_coords = my_project.get_coords(center="midbody", polar=False, align="nose", speed=0)
my_project_dists  = my_project.get_distances(speed=0)
my_project_angles = my_project.get_angles(speed=0)
my_project_areas = my_project.get_areas(speed=0)

With the adjusted bodyparts.
If I understand the website correctly, the command that you are showing is 'only' creating the graph of the skeleton (sanity check for further processing) - I adjusted the command and I can plot the graph (see image attached). It detected 2 mice but also a square. Could this be the problem?

When I am trying to run the unsupervised annotation, I still get the same error:
code:
supervised_annot = my_project.supervised_annotation()
error:
>>> supervised_annot = my_project.supervised_annotation()
Traceback (most recent call last):
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\indexes\base.py", line 3802, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas\_libs\index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas\_libs\hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'individual1_Center'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\deepof\data.py", line 1765, in supervised_annotation
    coords = self.get_coords(center=center, align=align)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\deepof\data.py", line 1124, in get_coords
    value[aid + ("_" if aid != "" else "") + center][coord_1],
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\frame.py", line 3806, in __getitem__
    return self._getitem_multilevel(key)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\frame.py", line 3857, in _getitem_multilevel
    loc = self.columns.get_loc(key)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\indexes\multi.py", line 2916, in get_loc
    loc = self._get_level_indexer(key, level=0)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\indexes\multi.py", line 3263, in _get_level_indexer
    idx = self._get_loc_single_level_index(level_index, key)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\indexes\multi.py", line 2849, in _get_loc_single_level_index
    return level_index.get_loc(key)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\indexes\base.py", line 3804, in get_loc
    raise KeyError(key) from err
KeyError: 'individual1_Center'

My second question is the following: if the problem is arising from the fact that my custom labeling has 8 bodyparts like the deepof_8, could I add the line 'exclude_bodyparts=[ "Tailtip"] to reduce it to 7 body parts?
I hope this is clear - thank you very much for your help!
Best wishes,
Julia

Hi,
Yes, first the get_graph_dataset function creates a few things, among others a matrix containing the connected body parts (this is what you plotted here) and a preprocessed block of data that can then be later used for the unsupervised learning pipeline.
That it plots both mice + this square is weird. Are you sure that you included the animal id in the command (as animal_id="individual1"?
Try the following commands in a clean environment:
import copy
import os
import numpy as np
import pickle
import deepof.data

from IPython import display
from networkx import Graph, draw
import deepof.visuals
import matplotlib.pyplot as plt
import seaborn as sns

deepof_custom = {
                "nose": ["leftear", "rightear"],
                "midbody": ["leftear", "rightear", "leftside", "rightside", "tailbase"],
                "tailbase": ["tailtip"],
            }

my_deepof_project = deepof.data.Project(
    project_path=os.path.join("[your_folder_name]"),
    video_path=os.path.join("[your_video_folder_name]"),
    table_path=os.path.join("[your_table_folder_name]"),
    project_name="[your_project_name",
    arena="polygonal-manual",
    animal_ids=["individual1", "individual2"],
    video_format=".mp4",
    bodypart_graph=deepof_custom,  # Can also be set to 'deepof_14' (default), or 'deepof_8'
    video_scale=380,
    iterative_imputation="partial",
    smooth_alpha=1,
    exp_conditions=None,
)

my_deepof_project = my_deepof_project.create(force=True)

(
    graph_preprocessed_coords,
    adj_matrix,
    to_preprocess,
    global_scaler,
) = my_deepof_project.get_graph_dataset(
    animal_id="individual1",  # Comment out for multi-animal embeddings
    center="midbody",
    align="midbody",
    window_size=25,
    window_step=1,
    test_videos=1,
    preprocess=True,
    scale="standard",
)

plt.figure(figsize=(3, 3))
draw(Graph(adj_matrix))
plt.show()

This should result in a graph plot of only one of your mice. From there on you than then train an unsupervised model, if this is what you want to do. For starters the supervised pipeline may be a little bit more interesting for you though.

I run the commands for individual 1and now there is just one mouse on the graph. However, when I run the supervised (sorry, misspelled it before) annotation, I get the same error (KeyError: 'individual1_Center'). I don't understand why it is not keeping the custom label?
Do you think I can change the bodyparts in my DLC files to the same labeling as the deepof_8? I don't know yet how such a code would look like but maybe it is easier to change these labels?
Just for me to know:
Are these commands important or not?
my_project_coords = my_project.get_coords(center="midbody", polar=False, align="nose", speed=0)
my_project_dists  = my_project.get_distances(speed=0)
my_project_angles = my_project.get_angles(speed=0)
my_project_areas = my_project.get_areas(speed=0)

Thank you so much for your help!
Best,
Julia

Hi Julia,
The supervised annotation has optional input arguments that would allow you to rename the names of the bodypart that represents the center to the one you use (similar for the align default that is Spine_1).
    def supervised_annotation(
        self,
        params: Dict = {},
        center: str = "Center",
        align: str = "Spine_1",
        video_output: bool = False,
        frame_limit: int = np.inf,
        debug: bool = False,
        n_jobs: int = 1,
        propagate_labels: bool = False,
    ) -> table_dict:

However, before you can call the supervised annotation you first would have to load a csv file that contains your grouping data (which mice were part of which experiment e.g. control and stress etc.)
In general you also can just change the column names in your tables which would mean that you can skip adding the renamed labels as additional input information.
Best regards,
Patrick

Hi Patrick
I have added the experimental conditions. For the code that you send in your last comment: I know I have to change 'center' to midbody, but how else would I need to complete the code?
Thank you very much :)

I just checked again and unfortunately some of the bodypart names in the supervised annotations remain hardcoded. To really use everything you would have to rename the table column headers to match the bodypart names that deepof uses. You could do that with a relatively simple script. The "correct" names would be the same ones you can find in the deepof_8 bodypart graph:
"deepof_8": {
    "Nose": ["Left_ear", "Right_ear"],
    "Center": [
        "Left_fhip",
        "Right_fhip",
        "Tail_base",
        "Left_ear",
        "Right_ear",
    ],
    "Tail_base": ["Tail_tip"],
},

so nose -> Nose, midbody -> Center etc.
I just added updating that section to also fully work with custom graphs to my to-do list.

Hi Patrick
I've changed now the bodypart labeling to deepof_8. Unfortunately, when I create the project I get the following error now:
>>> my_project = my_project.create(force=True)
Setting up project directories...
Loading trajectories...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\deepof\data.py", line 782, in create
    tables, quality = self.load_tables(verbose)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\deepof\data.py", line 416, in load_tables
    tab_copy = tab_copy.iloc[2:].astype(float)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\generic.py", line 6240, in astype
    new_data = self._mgr.astype(dtype=dtype, copy=copy, errors=errors)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\internals\managers.py", line 448, in astype
    return self.apply("astype", dtype=dtype, copy=copy, errors=errors)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\internals\managers.py", line 352, in apply
    applied = getattr(b, f)(**kwargs)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\internals\blocks.py", line 526, in astype
    new_values = astype_array_safe(values, dtype, copy=copy, errors=errors)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\dtypes\astype.py", line 299, in astype_array_safe
    new_values = astype_array(values, dtype, copy=copy)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\dtypes\astype.py", line 230, in astype_array
    values = astype_nansafe(values, dtype, copy=copy)
  File "C:\Users\odejul00\.conda\envs\deepof\lib\site-packages\pandas\core\dtypes\astype.py", line 170, in astype_nansafe
    return arr.astype(dtype, copy=True)
ValueError: could not convert string to float: 'x'

I am attaching here one of my csv files. I couldn't figure out where the error is coming from.... do you know?
75799_1DLC_dlcrnetms5_SRDec16shuffle3_200000_el.csv
Thank you very much for your help :)
Best,
Julia

Hi Julia,
There are two issues with your table: First you apparently accidentally included an additional row at the top of the table. Just remove it:

Next, the bodyparts "Left_fhip" and "Right_fhip" are missing the "f" (for "front" as in bigger labelling sets there is also a "bhip"
So if you remove that first row and rename "Left_hip" to "Left_fhip" and "Right_hip" to "Right_fhip" you should be able to create a project and run the supervised annotation to then create plots such as:

Note that the huddle annotator cannot be calculated due to a the 8-part scheme (it needs at least 11).
I hope this did help
Best regards,
Patrick

Cool! It worked out :) Thank you so much :)


Transaction with scammers

I make a 6 ton transaction with scammers.How can I return back my fund

@Habibut7  It sounds like your query would be best dealt with by the support team.
I'll have to refer you to the official support live chat with the ticket ID TON07638
Please see the link below to our dedicated support line:
Ton live support
Note; Click on the live chat icon at the bottom corner of the page to initiate chat with a live support agent


Moving to official consul image

There's now an official consul image: https://hub.docker.com/_/consul/
Let's see if our integration tests work with that.
Signed-off-by: Nishant Totla<EMAIL_ADDRESS>

Nope, doesn't really seem to work, not yet sure why.

@nishanttotla thought about doing the migration, yet I'm not sure this is quite "the same" image, might require some tweaks to make it work.
Logs are throwing: cannot set or renew session for ttl, unable to operate on sessions
My guess is that the config passed to the image (which shrinks the minimum ttl value to 1s) is not interpreted and thus operations on session gets rejected because the minimum ttl must be 10s. Will look into it and commit on top.

Yeah, looks like it has a really different entrypoint script, so guess it expects different arguments being passed

@abronan makes sense. Hopefully it's not too hard to do (the older image is fine too for now)

Nice~


Implement Output Channel for F# Language Service

@KevinRansom  In VFPT we have an output channel that we can enable in "diagnostic mode" to pump information onto while working on new features and fixing bugs. I think it'd be really useful for the F# vsintegration to have something similar. Even if it only works in debug mode and doesn't get shipped as part of the release.

@vasily-kirichenko or I could implement this

Yes please

Yes - I've done the same thing in Roslyn for Lightweight Solution Load here.

@cloudRoutine
This sounds like an excellent notion.

Fixed


Weird mouse behavior on visual studio  code

We have written the needed data into your clipboard because it was too large to send. Please paste.
Issue Type: Performance Issue
I am unable to use CTRL+V for paste and other keyboard shortcut such as copy,cut. Whenever i use these shortcuts nothing happens .sometimes it changes text randomly such as everytime I type Ctrl+X for cut, it reduces variable value in the selection.How can i solve this?
VS Code version: Code 1.48.2 (a0479759d6e9ea56afa657e454193f72aef85bd0, 2020-08-25T10:13:11.295Z)
OS version: Windows_NT x64 10.0.18363

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz (8 x 1800)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: enabledrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
7.89GB (4.18GB free)

Process Argv

Screen Reader
no

VM
0%

Process Info
CPU %	Mem MB	   PID	Process
    0	    77	   444	code main
    0	    27	  2780	   utility
    0	   184	  2900	   gpu-process
    0	   173	  3828	   window (style.css - css pracctive - Visual Studio Code)
    0	    13	  2968	     watcherService
    0	    11	  6780	       console-window-host (Windows internal process)
    0	    35	  5860	     searchService
    0	   115	  7420	     extensionHost
    0	    44	  7668	       electron_node agent.js
    0	    33	  8708	       "C:\Users\Shahab Uddin\AppData\Local\Programs\Microsoft VS Code\Code.exe" "c:\Users\Shahab Uddin\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\css-language-features\server\dist\node\cssServerMain" --node-ipc --clientProcessId=7420
    0	    55	  9964	       "C:\Users\Shahab Uddin\AppData\Local\Programs\Microsoft VS Code\Code.exe" "c:\Users\Shahab Uddin\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\html-language-features\server\dist\node\htmlServerMain" --node-ipc --clientProcessId=7420
    0	    27	  9996	       electron_node eslintServer.js
    0	    28	 10028	       electron_node serverMain.js
    0	    14	  6028	   electron-crash-reporter
    0	    66	  8032	   window (Issue Reporter)
    0	    79	  8256	   shared-process

Workspace Info
|  Window (style.css - css pracctive - Visual Studio Code)
|    Folder (css pracctive): 28 files
|      File types: jpg(14) svg(7) webp(1) png(1) html(1) js(1) mp3(1) css(1)
|      Conf files:;

Extensions (19)

Extension
Author (truncated)
Version

html-snippets
abu
0.2.1

emojisense
bie
0.7.0

path-intellisense
chr
2.2.1

codestream
Cod
8.3.3

hex-to-rgba
dak
1.0.0

vscode-eslint
dba
2.1.8

es7-react-js-snippets
dsz
2.8.3

vsc-material-theme
Equ
32.8.0

vsc-material-theme-icons
equ
1.2.0

prettier-vscode
esb
5.2.1

auto-rename-tag
for
0.1.4

auto-comment-blocks
kev
1.0.1

vscode-duplicate
mrm
1.2.1

LiveServer
rit
5.6.1

js-snippets
run
0.2.6

vim
vsc
1.16.0

quokka-vscode
Wal
1.0.315

JavaScriptSnippets
xab
1.8.0

html-css-class-completion
Zig
1.19.0

(1 theme extensions excluded)

Does the problem also happen if you reload with all extensions disabled? There is a Command Palette item that does this.

Does the problem also happen if you reload with all extensions disabled? There is a Command Palette item that does this.

When i selected "Reload with extensions disabled", this problem solved. Problem gone. But I need to use extensions.

/causedByExtension
Disable all extensions. Then enable them one at a time until you find the one causing your problem. Then report it to the extension author.

 There was an extension named "Vim" was causing the problem. Thanks a lot guys for your help.


Changes to root files cause rebuild of entire maven graph

Hello there,
I noticed an issue in the Nx affected graph. If I have a pom.xml at the repo root and I modify any other file (e.g. README.MD) in the root I get an affected graph of all the maven projects. In this case I have no project.json at the root.
I am using:
nx: 16.7.2
@jnxplus/nx-boot-maven: 7.3.0
Might have crept in when making the change to include parent poms as dependencies. I would be ok with the root pom being only an aggregator pom and not being part of the graph.
My root pom looks like:
<?xml version="1.0" encoding="UTF-8" ?>
<project
  xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"
>
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.foo.techradar</groupId>
  <artifactId>workspace</artifactId>
  <version>0.0.0</version>
  <packaging>pom</packaging>
  <modules>
    <module>libs/techradar</module>
  </modules>
</project>

Thoughts?

Hi @jbadeau
Did you try to add this files to .nxignore : https://nx.dev/reference/nxignore

We've experienced the same. It works perfectly for us if we disable addProjects with the following patch:
diff --git<EMAIL_ADDRESS><EMAIL_ADDRESS>index 134a380..b5a9e3e 100644
---<EMAIL_ADDRESS>+++<EMAIL_ADDRESS>@@ -24,7 +24,7 @@ pluginName) {
         encoding: 'utf-8',
     });
     const projects = JSON.parse(fs.readFileSync(outputFile, 'utf8'));
-    addProjects(builder, context, projects);
+    // addProjects(builder, context, projects);
     addDependencies(builder, projects);
 }
 exports.addProjectsAndDependenciesFromTask = addProjectsAndDependenciesFromTask;

Is there a reason the projects are added even though if they don't have a project.json? It also breaks by the way in the nx graph.

@dpnolte, I add the project root to the graph, so if we change Gradle config files, we need to rebuild all Gradle projects.

I’ll try the nxignore later today
On Wed, 6 Sep 2023 at 15:50, khalilou88 @.***> wrote:

@dpnolte https://github.com/dpnolte, I add the project root to the
graph, so if we change Gradle config files, we need to rebuild all Gradle
projects.
—
Reply to this email directly, view it on GitHub
https://github.com/khalilou88/jnxplus/issues/423#issuecomment-1708416397,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABFJO3EYU6KQC352FSYLXDXZB5UBANCNFSM6AAAAAA4J6K5NM
.
You are receiving this because you were mentioned.Message ID:
@.***>

@dpnolte, I add the project root to the graph, so if we change Gradle config files, we need to rebuild all Gradle projects.

Ah right. I understand your point. The downside is though that now a unrelated change (like a change in the README file) can cause all the gradle projects to be affected. I also noticed that it breaks the prompts when running stuff like nx g executor with the error "Cannot find configuration for '{rootProject}'".
Jsut thinking out loud here,  but wouldn't it be an option to add the related files as named inputs to gradle projects? Something like:
"{workspaceRoot}/*.gradle.kts",
"{workspaceRoot}/*.gradle",
"{workspaceRoot}/gradle.properties",

What do you think?

I wound def prefer named inputs. Forcing entire graph to rebuild is very important expensive.

@jbadeau for maven projects it's different, we need to add root project to the graph with build executor (mvn install -N). Otherwise it's will not work. See this old issue : https://github.com/khalilou88/jnxplus/issues/20
In older version I ran mvn install -N to build the root project first then I build other apps and libs. Thanks to the graph, nx ran this command now.
For Gradle we don't need to build the root project

@dpnolte I don't want to add same config to all projects. If the projectRoot is the problem (. for root project). I can use instead of . the Gradle wrapper folder. Then add config files as named inputs just to the root project when building the graph.

@dpnolte just checked Inputs could be set in nx.json, so planning to remove the root project from the the graph for Gradle projects

What about maven? The .nxignore breaks the build as it’s looking for the root Pom.

@jbadeau I will investigate changing the projectRoot for root project and add the root pom as named input and remove it from files in this part of code :
    const projectSourceFile = joinPathFragments(projectRoot, 'pom.xml');
    //TODO fix hash
    const fileData: FileData[] = [{ file: projectSourceFile, hash: 'abc' }];
    context.fileMap[artifactId] = fileData;

But before I need to create a reproduction with smoke tests to be sure that I will correct this issue.

Alrighty.

For information, before looking at this issue, we need to update code to use NxPluginV2

@jbadeau It's the new way to add projects to the NX Project Graph : https://nx.dev/packages/devkit/documents/NxPluginV2#type-alias-nxpluginv2.

Hi @jbadeau I am working to make project.json mandatory for all maven projects. With this feature you will have freedom to configure the nx workspace with your needs with named input.
The project.json for the maven root project will be in the the wrapper folder.

That’s great news.
A few questions

Does it make sense to put the project.json in the wrapper folder? Going forward, it would prob make sense to remove the wrapper or make it optional as modern ci tooling provide binaries directly.
Why not put the project.json in a project folder under libs with the pom.xml?
Isnt named inputs for determining the inputs for target caching and not the affected which is determined something like commit?

I currently have a pom.xml and a project.json at the root and I ignore almost everything at toot using .nxignore. If I could move the parent pom to any folder that would be ideal.
im pretty flexible

It's not possible to have pom.xml in a sub folder without moving all maven projects into that folder.
Maven wrapper is the recommended way
There is two root projects : maven root project and nx root project, put project.json in the wrapper will help to not merge the configuration of this two projects.
ignore almost everything using .nxignore is not good because maybe you have front end projects
if you create a project.json at the root folder this should ignore maven config files

I upgraded to 16.7.4 of Nx and I see that whenever I change any Java code in a app module, it is building all apps. Previously it used to work fine. Not sure where I did the mistake.

nx affected is building all Spring boot projects no matter which app file is changed.

nx:affected graph is showing only one project, but when I am running in the Azure pipeline it is building all spring boot projects.

@nosaku it was working before in azure ? I think you miss something to make azure work with Nx?

@nosaku it was working before in azure ? I think you miss something to make azure work with Nx?

Yes it was working fine previously then I did "nx repair" as I saw some warnings. Not sure if it got broken after this. I am trying with a separate branch now.

@nosaku it was working before in azure ? I think you miss something to make azure work with Nx?

Yes it was working fine previously then I did "nx repair" as I saw some warnings. Not sure if it got broken after this. I am trying with a separate branch now.

I create package-lock manually. Does it make any difference?
npm install --no-save
npm install --package-lock
I did this long ago when "nx" was considering package.json always changes when doing "npm install" as the timestamp was getting updated with a "blank line" and triggering all builds. But this I did very long ago.

just changed the .mvn/jvm.config and pushed the changes and it built all the spring boot apps. To build affected apps I am using below which was working fine till recently.
nx affected --target=build --parallel=3 --base=HEAD~1 --configuration=train

Added .nxignore and copied all .gitignore stuff there including package-lock.json. Still nx affected builds all Spring boot apps... it works fine if I change NestJS or Angular apps and only those are built. But along with them all spring boot apps are built too. And if I change only package.json or any other spring boot app file, it builds all spring boot apps.

Are you sure you have no root changes between in your base and head

Yes no changes.. I just tried by changing one source file in one spring boot app.

When I run the below in Azure pipeline, I always get all Spring boot apps... but when I run locally I get only the changed app.
nx show projects --affected --base=HEAD~1

Does it make any difference with local being Windows and pipeline being self hosted Ubuntu?

I tried in a Linux machine and nx affected shows all spring boot apps... the same in Windows show only the one which is affected.

I don't think it's a good idea to put  pom.xml and project.json  in different locations for the same project (root project).
so I am trying to create maven and gradle projects in a sub folder. I beleive it's the best solution

this issue is more nx than nx-maven or nx-gradle. But we are working to make life easier for our users. so i added two issue to generate maven and gradle projects in a subfolders.
maven : https://github.com/khalilou88/jnxplus/issues/535
gradle : https://github.com/khalilou88/jnxplus/issues/534
closing this issue


Stack unwinding for Android 8.0 and above is not supported

Stack unwinding is not supported for android 8.0 and above as of now.
Please, keep this issue open as a placeholder for people to subscribe to.

Android 8.0 and 9.0 coverage is surely in plan.

I added 8.x support in daed61c01f68966415460821dab1bdca6f2afcee and 94eb24c53a051aa4eec18a8b738fccac2baabe9d.
9.x is yet to come, but I'll close this issue now, as it is only tracking 8.0.


Reverse logic of :password required allows empty updates.

You require both password and password_confirmation, yet if a user supplies only a password_confirmation then update actions do not fail like they would if password was supplied without password_confirmation.

That never was and never will be the case for me but this ticket is worthless it's so old.


Snakemake waiting to finish all parallel jobs before starting next parallel job

5.30.1

I wrote a simple Snakemake rule which runs a program on all files in a folder. This rule is running the program in parallel on n-number of files, where n is CPU cores given to Snakemake (e.g. -j 4). The rule starts with running n jobs in parallel and it waits until each one of them is finished until starting a new "batch" of n jobs. The rule is very inefficient CPU-time-wise, when execution time varies heavily between each file in the folder as most cores are waiting all jobs to finish.

Example rule:
CLUSTER_IDS, = glob_wildcards(os.path.join(WORK_DIR, "fasta", "{id}.fasta"))
rule msa:
    input:
       expand(os.path.join(WORK_DIR, "msa", "{id}.afa"), id=CLUSTER_IDS)

rule:
    input:
        os.path.join(WORK_DIR, "fasta", "{id}.fasta")
    output:
        os.path.join(WORK_DIR, "msa", "{id}.afa")
    shell:
        "{MUSCLE_PATH}/muscle3.8.31_i86darwin64 -in {input} -out {output}"

Example execution flow with 4 parallel jobs:
job1|-----           |job5|-----     |...|->
job2|---             |job6|--------  |...|->
job3|----------------|job7|--        |...|->
job4|-               |job8|----------|...|->

Originally asked: https://stackoverflow.com/questions/68541814/snakemake-waiting-to-finish-all-parallel-jobs-before-starting-next-parallel-job

Updated Snakemake to version 6.5.2 and the issue is gone!


Is setting apt-mirrors for ubuntu required to run this role?

I might misunderstand something, so please correct me if I'm wrong.
The task configures an apt-mirror for ubuntu. This is quite nonstandard and breaks my own setup, where I have a "base" role that configures apt on all systems according to our guidelines and setups.
One of that is to set the mirrors.
So, it seems to a bot out of place in this role. AFAIK the role works fine without setting mirrors on Ubuntu, but I did not test that.
If so, should this step not be removed instead? And if so, please let me know and I'll mock up a PR for that.

Jitsi needs access to Ubuntu's universe repository. Maybe there's a better way to solve this. But that's the reason it's in the role.

I now understand the reasoning better. My VPS has that enabled by default, so in my case it was a bit redundant, but not breaking stuff either.
Guess that closes this issue.


build: bump semantic-release to 17.3.9 to address dependabot alert

Ticket Number
UCD-1697
Description of change
bump semantic-release to 17.3.9 to address dependabot alert - https://github.com/foundry4/appeal-planning-decision/security/dependabot/package-lock.json/marked/open
Checklist

[ ] Requires infrastructure changes
[ ] If adding or remove environment variables (e.g. in docker-compose.yaml) then I have updated the appropriate Helm chart
[ ] I have updated the documentation accordingly
[ ] My commit history in this PR is linear
[ ] New features have tests
[ ] Breaking change (team conversation required)

Important
Please do not merge from master (please only rebase). This keeps the history linear and easier to debug.

close, unwanted

I'm not sure this is unwanted, an outdated package shouldn't ever be ignored I don't think regardless of what it's currently used for. Can you reopen?

@marcfielding1 this isn't really used in the source code. Semantic Release is used to generate the version numbers from the commit messages, so it never goes anywhere near the production code - it's a dev dependency only

@MrSimonEmms Yeah I do understand that, I guess my point would be if we're delivering software to the client and dependency that's way of date sends the wrong message - also whilst this dep likely won't cause us any issues if we adopt that as a policy then eventually it could happen that we get outdated deps creeping in.
I do agree with you, but I disagree that we shouldn't fix it because it's outdated.


Canvas Live Reload from changed file - Development Tool (fluentAPI)

I am looking to build a development plugin for the Camunda Fluent API, which will continually output the bpmn xml of a fluent API build.
I was going to build a renderer, but this seemed a better thing to add directly into modeler given all of the tools are already built into it.
What would it take to have a "live reload" type of feature in modeler:
Currently when the source xml changes the modeler will ask if you want to reload the file, this is a good thing given the possibility of accidental changes (i know i have done this many times, and the modal saved me, allowing me to save ;) ).  But what about if we want the automatic reload without the warning/modal popup?
Is this something that could be added as a optional feature in the menu to enable something like "File Change Auto Reload" ?
Thanks!

Why exactly would you like to life-reload the diagram? To perform modification in the modeler later? To visualize the modeled diagram?

As I update the fluent API, I want the diagram to auto reload to show the output.  The fluent APi code would be exporting a bpmn file that the modeler would have open

So you'd like to get a live preview of the things you're building using a DSL.

@nikku It could be from the DSL / Fluent API.  But thats not 100% the use case in every scenario.
Right now, the modeler will detect if a reload is applicable and ask the user.  The extra configurable option would be to add a option that can be turned on that will auto reload when the change is detected.
This could be for checking the output of a Fluent API build, it could be for working with an existing  model and making changes to it, etc.
It is not just about a "preview", it is about being able to inspect the various elements of the outputted build.

Moving to backlog, as a quick-view / reload mode seems like a valid feature request.

Could be implemented via a confirmation ("don't ask again") in the reload notification message, cf. https://www.electronjs.org/docs/api/dialog#dialogshowmessageboxbrowserwindow-options.


Update dependency ssh2 >= v1.4.0

Package ssh2 had OS Command Injection bug in versions previous to v0.8.5: https://github.com/advisories/GHSA-652h-xwhf-q4h6

up


Add TaskScheduler

New Feature: Task Scheduler for Function Execution
This pull request introduces a new functionality called "Task Scheduler" by introducing a completely new class called TaskScheduler. This class empowers users to schedule the execution of functions, seamlessly handling both synchronous and asynchronous tasks.
Key Changes:

TaskScheduler Class: The new TaskScheduler class has been created to manage task scheduling and execution.

add_task Method: The TaskScheduler class now includes a add_task method. This method allows users to schedule the execution of a function by providing a func parameter to represent the function to be executed, and execute_at datetime parameter for future execution, and optional args and kwargs parameters for passing function arguments and keyword arguments.

Example Usage:
python
Importing the required dependency
from fastapi import get_scheduler
Creating an async and a sync function to be used in scheduling
async def sample_async_task(name: str):
    print(f"Executing task for {name} at {datetime.now()}")
    await asyncio.sleep(2)
    print("Execution Done")

def sample_sync_task(name: str):
    print(f"Executing task for {name} at {datetime.now()}")
    time.sleep(2)
    print("Execution Done")

Example Usage
@app.get("/")
async def schedule_async_task_endpoint(scheduler: TaskScheduler = Depends(get_scheduler)):

    # Get the current time
    current_time = datetime.now()

    # Calculate the time 3 seconds from now
    execute_at = current_time + timedelta(seconds=10)

    scheduler.add_task(sample_async_task, execute_at, args=("ali",))
    return {"message": "Task scheduled."}

@app.get("/2")
async def schedule_sync_task_endpoint(scheduler: TaskScheduler = Depends(get_scheduler)):

    # Get the current time
    current_time = datetime.now()

    # Calculate the time 3 seconds from now
    execute_at = current_time + timedelta(seconds=10)

    scheduler.add_task(sample_sync_task, execute_at, args=("ali",))
    return {"message": "Task scheduled."}

scheduler.add_task arguments
add_task(
function callable sync or async,
execution time,
args as tuple
kwargs as dict
)

This new Task Scheduler functionality is introduced through the TaskScheduler class, offering users a robust solution for scheduling and executing functions in a variety of contexts, whether they are synchronous or asynchronous. This PR adds significant value to the project, enhancing its capabilities and enabling efficient task automation and concurrency management.

IMO
if your application is relatively simple, without any time-sensitive or background processing requirements, and doesn't involve complex workflows, a task scheduler might not be necessary (especially at the framework level).
You can manage simple periodic tasks using basic Python functionalities like the time.sleep() function or a simple cron job (if your application is deployed on Unix-like systems).
What the community use
In the Python ecosystem, there are several libraries available for task scheduling, such as schedule, Celery. These libraries can be integrated into your app easily, (IMO if you include TaskScheduler you will somehow move the architecture of Django, Batteries 🥲).
IMO, the decision to add a Scheduler back to Sebastian, but I can't see the value of creating something on the top of the framework level, for an integration that can be easy to add if we use some celery.

Thanks @achaayb! 🍰
I actually agree with @yezz123, this would probably belong outside of FastAPI.
There are a lot of things to have in mind for use cases like this, and it's probably better to have a tool like Celery, RQ, ARQ... or maybe Prefect, Dagster, etc.
Some alternatives you could try working on would be tutorials to integrate one of those tools. Or if you definitely want to use this approach, you could write a blog post about how to write this code as part of the app. Or you could even create a library to expose it if you want.
Given that, I'll close this one, but thanks for the effort! ☕


Display labels for Interpreters in the interpreter list do not seem to be right

I believe something changed recently that cause the display name to change.
I don't remember seeing packaged by conda forge earlier.

This could be due to the changes in Anaconda

Confirmed, not a bug.


C++ FinalClient::request small bug

This should be easy to reproduce, using latest fbec <IP_ADDRESS> with --cpp --final --proto
The generated FinalClient::request(..) methods has
size_t serialized = Sender::send(value);

which my gcc 11 compiler complains
Fixed by changing to
size_t serialized = FinalSender::send(value);

btw, really awesome library!

Thanks for reporting! Fixed in <IP_ADDRESS>


Create Architecture Documentation based on key Technologies

Write a technical Documentation up on the key Technologies.
This includes both the Client-Server Architecture, SaaS Solutions we implement as well as key Libraries like tRPC or others.
Also add visualizations for relevant topics.

Done in other issue


Fixes #98: Three markup related minor fixes for the index.html

@all-contributors add @rpkoller for code


mantl and rancher

Anyone has heard rancher ，it seems promising too ,what is the difference between mantl and rancher?

Mantl is more a whole kitchen sink that provides a whole stack up/down for microservices. Rancher provides some of the same features that Mantl does but without the whole stack approach. For example, load balancing apps deployed in Rancher still require additional containers to be added as a service to a container. Mantl handles this automatically with it's all inclusive approach.
Rancher would be best compared to as a replacement for Docker Swarm + Docker UCP. That's just from my perspective though.


Adding IndexCreationStatistics struct to address #95

I tried to address #95 but the Github checks fail on me. This is weird since I didn't touch the code in aln.cpp at all.

This should work now.

Issue #95 is still open: Remember that you can write "Closes #95" in your commit message to have GitHub auto-close the referenced issue as soon as the commit is merged into main.

Right, I just removed these two lines in 0bbadec, I will close the issue manually this time.

Thanks!


fromHTML fails to add iFrame content

I've got an HTML page with a couple google charts loaded via iFrames. We'd like to make the page render all charts within the selected div of frames.

function makePDF()
{
    var doc = new jsPDF('p','pt','a4);
    doc.fromHTML($('#frames')[0],30,30, { 'width':1080 });
    doc.output('dataurlnewwindow');
}

The issue is that the iFrames within that div don't render on the PDF output. I double-checked the console debugger to make sure the jQuery reference was good as well. I also tried using rasterizeHTML (and of course switching to addHTML), but that apparently has a known iFrame limitation.
As a hack, I also tried changing the iFrames to embed links, but I got the same result. Any suggestions?
Edit: For yet another hack, I attempted to load the pages for each chart into divs using JQuery's .load(), but Google Charts refuses to operate that way.

Can we open the iframe content to new page/window and then save that new windows as pdf.
Anyone knows how to open iframe content to new windows and command new window to save as pdf using jspdf
Help plz


Update tsu

Could you change the description to match your change? Looks goog

Fixed in https://github.com/cswl/tsu/commit/b664e2662e9a23a6fdd305c8505dfbd233e3e33c


fix(sqllab): flaky json explore modal due to over-rendering

SUMMARY
This commit reopens #26156, which resolves the issue detailed in https://github.com/apache/superset/issues/26276 by excluding the changes in 'QueryAutoRefresh/index.tsx' and 'reducers/sqlLab.js'.
This commit also reverses the changes introduced in #18934. Now that the issue of overrendering due to redux has been resolved, it also effectively addresses the problem outlined in issue #26755.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before:
https://github.com/apache/superset/assets/1392866/8d81ba78-7142-438c-8c5c-b5049ef6ea94
After:
https://github.com/apache/superset/assets/1392866/c2023683-47f1-4a45-97c0-c14e4d7b6469
TESTING INSTRUCTIONS
Go to SQL Lab and run a query contains a json blob
Select a table in order to execute a table preview query
Click Results tab before the table preview query ends and then click the JSON column to open the JSON modal
Check the JSON modal persists while the async query status is updating
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up

cc @gaurav7261

/testenv up

@justinpark any reason not to merge this?


Router doesn't work from file:/// URLs

When loading a simple "hello world" redux-router app from a file:/// URL, you get a bunch of warnings and the app never loads:
Warning: Location "/home/eric/src/react-redux-boilerplate/build/index.html" did not match any routes
 app.js:22240:10

Warning: Failed propType: Required prop `location` was not specified in `RoutingContext`. Check the render method of `ReduxRouterContext`.
 app.js:3308:10

Warning: Failed propType: Required prop `routes` was not specified in `RoutingContext`. Check the render method of `ReduxRouterContext`.
 app.js:3308:10

Warning: Failed propType: Required prop `params` was not specified in `RoutingContext`. Check the render method of `ReduxRouterContext`.
 app.js:3308:10

Warning: Failed propType: Required prop `components` was not specified in `RoutingContext`. Check the render method of `ReduxRouterContext`.
 app.js:3308:10

Warning: Failed Context Types: Required child context `location` was not specified in `RoutingContext`. Check the render method of `ReduxRouterContext`.

Loading the same app from a simple node-static server with a http:// URL works fine.
Either this is a bug, and the router should work from file:/// URLs, or it's a "wontfix", but it should be documented in the README to save people headaches.

I just ran into the same issue, and the problem for me was that i was using
import createHistory from 'history/lib/createBrowserHistory';
instead of import createHistory from 'history/lib/createHashHistory'; on hindsight it makes sense that resources loaded from file do not work with browser history

This belongs on history anyway. Potentially a dupe of https://github.com/rackt/history/issues/126.


kairos-agent doesn't exit if upgrade fails

Seems that when we reach https://github.com/kairos-io/kairos-agent/blob/6110cae4e54ccd4e02ff3145f23a9d6f96736579/main.go#L155  the agent still exists with 0, so the failure is not catched up from scripts invoking it:
root@edge-813115b5e06247199a63913e9b1a8745:/sys/firmware/efi/efivars# kairos-agent --debug upgrade --source dir:/trusted-boot/
cannot upgrade from live media/unknown boot state

From the chat:
this is the output of upgrade cmd, it didn't throw out any error or exit with an error code. thats why we thought upgrade was successful but it actually didnt run at all (modificato)

The agent exits with 1 like it should:
~/workspace/kairos/kairos-agent (main)*$ ./build/kairos-agent upgrade; echo $?
cannot upgrade from live media/unknown boot state
1

The problem must be the suc upgrade script that doesn't set -e.

Thanks for checking @jimmykarily - things seems to work as expected, closing


Fork server died: Success

I run Eclipser in a docker container and running a target results in a crash of Eclipser:
# dotnet /Eclipser/build/Eclipser.dll -t 5 -s eclipser -o out -p fuzz-target --exectimeout 5000 --arg foo -v 2
[00:00:00:00] [*] Fuzz target : /targets/test/fuzz-target
Fork server died: Success

Unhandled Exception: System.Exception: Failed to initialize fork server for coverage tracer
   at Eclipser.Executor.initializeForkServer(FuzzOption opt) in /Eclipser/src/Core/Executor.fs:line 75
   at Eclipser.Executor.initialize(FuzzOption opt) in /Eclipser/src/Core/Executor.fs:line 92
   at Eclipser.Fuzz.main(String[] args) in /Eclipser/src/Fuzz/Fuzz.fs:line 137
Aborted (core dumped)

the target runs without problem when called native or when fuzzed via afl-fuzz -Q.
what could the issue be?

Thank you for reporting the issue. Could you upload the target binary you tested?

Thank you for reporting the issue. Could you upload the target binary you tested?

eclipser.zip
here you go!

eclipser.zip
here you go!

The current Dockerfile is based on Ubuntu 18.04, so glibc version mismatch makes fuzz-target to abort. I tried to directly execute the target program in our docker container, and encountered the following error.
test@7450e4731ddb:~/Eclipser$ dotnet build/Eclipser.dll -t 5 -o out -p fuzz-target --exectimeout 5000 --arg foo -v 2
[00:00:00:00] [*] Fuzz target : /home/test/Eclipser/fuzz-target
Fork server died: Success

Unhandled Exception: System.Exception: Failed to initialize fork server for coverage tracer
   at Eclipser.Executor.initializeForkServer(FuzzOption opt) in /home/test/Eclipser/src/Core/Executor.fs:line 75
   at Eclipser.Executor.initialize(FuzzOption opt) in /home/test/Eclipser/src/Core/Executor.fs:line 92
   at Eclipser.Fuzz.main(String[] args) in /home/test/Eclipser/src/Fuzz/Fuzz.fs:line 137

test@234c0ac61d54:~/Eclipser$ ./fuzz-target
./fuzz-target: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./fuzz-target)

test@234c0ac61d54:~/Eclipser$ ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1.4) 2.27

When I tried Ubuntu 20.04 instead of 18.04 (for this you can simply fix the two lines from the above Dockerfile), the fork server error has disappeared.
Would you confirm whether this was the problem in your case, too?

The current Dockerfile is based on Ubuntu 18.04, so glibc version mismatch makes fuzz-target to abort. I tried to directly execute the target program in our docker container, and encountered the following error.
test@7450e4731ddb:~/Eclipser$ dotnet build/Eclipser.dll -t 5 -o out -p fuzz-target --exectimeout 5000 --arg foo -v 2
[00:00:00:00] [*] Fuzz target : /home/test/Eclipser/fuzz-target
Fork server died: Success

Unhandled Exception: System.Exception: Failed to initialize fork server for coverage tracer
   at Eclipser.Executor.initializeForkServer(FuzzOption opt) in /home/test/Eclipser/src/Core/Executor.fs:line 75
   at Eclipser.Executor.initialize(FuzzOption opt) in /home/test/Eclipser/src/Core/Executor.fs:line 92
   at Eclipser.Fuzz.main(String[] args) in /home/test/Eclipser/src/Fuzz/Fuzz.fs:line 137

test@234c0ac61d54:~/Eclipser$ ./fuzz-target
./fuzz-target: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./fuzz-target)

test@234c0ac61d54:~/Eclipser$ ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1.4) 2.27

When I tried Ubuntu 20.04 instead of 18.04 (for this you can simply fix the two lines from the above Dockerfile), the fork server error has disappeared.
Would you confirm whether this was the problem in your case, too?

To continue, in Ubuntu 20.04, I encounter different error as follow.
test@6e07cf102579:~/Eclipser$ dotnet build/Eclipser.dll -t 5 -o out -p fuzz-target --exectimeout 5000 --arg foo -v 2
[00:00:00:00] [*] Fuzz target : /home/test/Eclipser/fuzz-target
[00:00:00:00] [*] Total 1 initial seeds
[00:00:00:00] [*] Time limit : 5 sec
[00:00:00:00] [*] Start fuzzing
[00:00:00:00] Fuzzing with:  ...16bytes... (0) (Right)
[00:00:00:00] [Warning] Unexpected operand size
[00:00:00:00] [Warning] Coverage logging failed : []
[00:00:00:00] [Warning] Coverage logging failed : []
[00:00:00:00] [Warning] Coverage logging failed : []

I assume this is because the target program is multi-processed or multi-threaded. Could you provide more information about the target program, like its source code?

To continue, in Ubuntu 20.04, I encounter different error as follow.
test@6e07cf102579:~/Eclipser$ dotnet build/Eclipser.dll -t 5 -o out -p fuzz-target --exectimeout 5000 --arg foo -v 2
[00:00:00:00] [*] Fuzz target : /home/test/Eclipser/fuzz-target
[00:00:00:00] [*] Total 1 initial seeds
[00:00:00:00] [*] Time limit : 5 sec
[00:00:00:00] [*] Start fuzzing
[00:00:00:00] Fuzzing with:  ...16bytes... (0) (Right)
[00:00:00:00] [Warning] Unexpected operand size
[00:00:00:00] [Warning] Coverage logging failed : []
[00:00:00:00] [Warning] Coverage logging failed : []
[00:00:00:00] [Warning] Coverage logging failed : []

I assume this is because the target program is multi-processed or multi-threaded. Could you provide more information about the target program, like its source code?

I will try to update the container to Ubuntu 20.04, thanks!
The source code is: https://gitlab.com/wg1/jpeg-xl, target is tools/djxl

I will try to update the container to Ubuntu 20.04, thanks!
The source code is: https://gitlab.com/wg1/jpeg-xl, target is tools/djxl

The current Dockerfile is based on Ubuntu 18.04, so glibc version mismatch makes fuzz-target to abort. I tried to directly execute the target program in our docker container, and encountered the following error.

can you supply that dockerfile? I tried to switch my Dockerfile to 20.04 and the correct microsoft deb, but the qemu compilation fails

The current Dockerfile is based on Ubuntu 18.04, so glibc version mismatch makes fuzz-target to abort. I tried to directly execute the target program in our docker container, and encountered the following error.

can you supply that dockerfile? I tried to switch my Dockerfile to 20.04 and the correct microsoft deb, but the qemu compilation fails

Oh, would you check if your Dockerfile trying to checkout 'v2.0' release of Eclipser?
Ubuntu 20.04 QEMU build issue was fixed in more recent commit, so you should use the latest commit in master branch.
(Today I updated the Dockerfile to use master branch instead of v2.0 release).

Oh, would you check if your Dockerfile trying to checkout 'v2.0' release of Eclipser?
Ubuntu 20.04 QEMU build issue was fixed in more recent commit, so you should use the latest commit in master branch.
(Today I updated the Dockerfile to use master branch instead of v2.0 release).

Dockerfile in both master in dev are Using Ubuntu 18.04 - just checked again here on github :)
https://github.com/SoftSec-KAIST/Eclipser/blob/master/Dockerfile
https://github.com/SoftSec-KAIST/Eclipser/blob/dev/Dockerfile

Dockerfile in both master in dev are Using Ubuntu 18.04 - just checked again here on github :)
https://github.com/SoftSec-KAIST/Eclipser/blob/master/Dockerfile
https://github.com/SoftSec-KAIST/Eclipser/blob/dev/Dockerfile

Sorry, it seems my words were confusing.
What I meant was to checkout master branch as in https://github.com/SoftSec-KAIST/Eclipser/blob/master/Dockerfile , and the version string ("18.04") must be replaced.
I fixed the two lines of that Dockerfile as follow, and it compiled without an error.
jschoi@ubuntu:~/Eclipser$ git diff Dockerfile
diff --git a/Dockerfile b/Dockerfile
index ba1d341..729b3fc 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,5 +1,5 @@
 # For other Ubuntu versions, you should also fix the URL of .NET Core install
-FROM ubuntu:18.04
+FROM ubuntu:20.04

 WORKDIR /root/

@@ -18,7 +18,7 @@ RUN apt-get update && \
 RUN apt-get -yy build-dep qemu

 # Install .NET Core for Eclipser
-RUN wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
+RUN wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
     dpkg -i packages-microsoft-prod.deb && \
     apt-get update && apt-get -yy install dotnet-sdk-2.1 && \
     rm -f packages-microsoft-prod.deb

Sorry, it seems my words were confusing.
What I meant was to checkout master branch as in https://github.com/SoftSec-KAIST/Eclipser/blob/master/Dockerfile , and the version string ("18.04") must be replaced.
I fixed the two lines of that Dockerfile as follow, and it compiled without an error.
jschoi@ubuntu:~/Eclipser$ git diff Dockerfile
diff --git a/Dockerfile b/Dockerfile
index ba1d341..729b3fc 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,5 +1,5 @@
 # For other Ubuntu versions, you should also fix the URL of .NET Core install
-FROM ubuntu:18.04
+FROM ubuntu:20.04

 WORKDIR /root/

@@ -18,7 +18,7 @@ RUN apt-get update && \
 RUN apt-get -yy build-dep qemu

 # Install .NET Core for Eclipser
-RUN wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
+RUN wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
     dpkg -i packages-microsoft-prod.deb && \
     apt-get update && apt-get -yy install dotnet-sdk-2.1 && \
     rm -f packages-microsoft-prod.deb

yes, works. I had inserted a bug in my dockerfile thats why it didnt worked.
any progress in the threading issue that kills eclipser? because it works fine with afl-fuzz -Q and symqemu.

yes, works. I had inserted a bug in my dockerfile thats why it didnt worked.
any progress in the threading issue that kills eclipser? because it works fine with afl-fuzz -Q and symqemu.

Unfortunately, I think it will take some time until I can fix this issue :(
Eclipser's instrumentation had to handle fork() syscalls in a different way from AFL-QEMU, and I suspect there are some bugs in this code. This may require some debugging and testing.

no hurry, just so that it is not forgotten :)


help running  tutorial at examples/MMPT

I'm trying to run the simple tutorial given here: https://github.com/pytorch/fairseq/tree/main/examples/MMPT .
Unfortunately, I get Omegaconf exceptions.
What I have tried:
python3 -m venv testenv
source testenv/bin/activate
now, testenv is activated
git clone https://github.com/pytorch/fairseq
cd fairseq
pip install -e .
export MKL_THREADING_LAYER=GNU
cd examples/MMPT
pip install -e .
pip install transformers==3.4
mkdir pretrained_models  //  pretrained_models located under MMPT/..
cd pretrained_models; wget https://www.rocq.inria.fr/cluster-willow/amiech/howto100m/s3d_howto100m.pth
wget https://www.rocq.inria.fr/cluster-willow/amiech/howto100m/s3d_dict.npy
now I have:
>> ls pretrained_models/
s3d_howto100m.pth
s3d_dict.npy

under examples/MMPT, I run:
mkdir -p runs/retri/videoclip
cd runs/retri/videoclip; wget https://dl.fbaipublicfiles.com/MMPT/retri/videoclip/checkpoint_best.pt

>> ls runs/retri/videoclip/
checkpoint_best.pt.1

Finally,
python locallaunch.py projects/retri/videoclip.yaml --dryrun
and I get:
Traceback (most recent call last):
File "locallaunch.py", line 148, in
main(args)
File "locallaunch.py", line 128, in main
pipelines = [Pipeline(fn) for fn in args.yamls.split(",")]
File "locallaunch.py", line 128, in
pipelines = [Pipeline(fn) for fn in args.yamls.split(",")]
File "locallaunch.py", line 54, in init
job_config = recursive_config(fn)
File "..../fairseq/examples/MMPT/mmpt/utils/load_config.py", line 61, in recursive_config
base_config = recursive_config(includes)
File "..../fairseq/examples/MMPT/mmpt/utils/load_config.py", line 62, in recursive_config
config = OmegpretrainedaConf.merge(base_config, config)
NameError: name 'OmegpretrainedaConf' is not defined
I tried upgrading omegaconf:
pip install omegaconf==2.1.1
but now after running:
Traceback (most recent call last):
File "locallaunch.py", line 148, in
main(args)
File "locallaunch.py", line 128, in main
pipelines = [Pipeline(fn) for fn in args.yamls.split(",")]
File "locallaunch.py", line 128, in
pipelines = [Pipeline(fn) for fn in args.yamls.split(",")]
File "locallaunch.py", line 54, in init
job_config = recursive_config(fn)
.......
......
omegaconf.errors.ConfigAttributeError: Missing key includes
full_key: includes
object_type=dict
my os is linux, python 3.7.3 and installation was described above.
Probably I'm doing something wrong.
Any help would be much appreciated!

I have no idea why, ran it today again on some existing venv and it succeeded. For anybody it may help in the future:
on the existing venv (where the code worked):
python version is 3.7.3
omegaconf is 2.0.6
torch version is 1.9.0+cu102

Had similar issue when running python locallaunch.py projects/retri/videoclip.yaml --dryrun. Got NameError: name 'BertEmbeddings' is not defined. I just unwrapped transforms imports in mm.py file from try except close and it worked

Hi @yonatanshemeshhuji , do you know that how can I download data/youcook/youcook_val.pkl ?


Replace call to vlime#Getcurpos with getcurpos

The inspector should now be fully functional again.

Yeah, right. Thanks!


Ensure Polymer 3 Modulizer compatibility

See “Generic steps” at vaadin/components-team-tasks#326

I have tried the --add-import-path flag, which sets the static getter on the element:
  static get importPath() {
    return import.meta.url;
  }

As of now, this seems not to be supported by polyserve transform, and therefore breaks in all browsers, except for Chrome which supports it starting from the 64 stable release.

Closing this for now, we will have to revisit this once the importPath gets proper support.


ENOENT lstat  on workspace

[*] 2713 field calls mapped
[INFO] Scanning default path : /home/lamba/workspace/org.telegram.messenger/dex
internal/fs/utils.js:220
    throw err;
    ^

Error: ENOENT: no such file or directory, lstat '/home/lamba/workspace/org.telegram.messenger/dex'
    at Object.lstatSync (fs.js:906:3)
    at Object.forEachFileOf (/home/lamba/dexcalibur/src/Utils.js:86:66)
    at Analyzer.path (/home/lamba/dexcalibur/src/Analyzer.js:892:12)
    at Project.fullscan (/home/lamba/dexcalibur/src/Project.js:381:22)
    at Object.<anonymous> (/home/lamba/dexcalibur/dexcalibur:169:37)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    at Module.load (internal/modules/cjs/loader.js:811:32)
    at Function.Module._load (internal/modules/cjs/loader.js:723:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10) {
  errno: -2,
  syscall: 'lstat',
  code: 'ENOENT',
  path: '/home/lamba/workspace/org.telegram.messenger/dex'
}

Hello,
how did you launch dexcalibur ? What is your command ?
Is apktool is configured into config.js ?

1 - ./dexcalibur --app=org.telegram.messenger --port=8000 --pull
2 - ./dexcalibur --app=org.telegram.messenger --port=8000
Yes I got the path to apk as follows: /home/lamba/Desktop/apktool.jar
I got it pointed to the jar file directly, I assume theres no problem with that.

Can you test with "java --jar /home/lamba/Desktop/apktool.jar" into "apktPath" into config file ?

Didnt work either. I remain with the same bug

This error occurs when APK file pull from device has not been successfully extracted with APKTool.
Can you remove "/home/lamba/workspace/org.telegram.messenger/" from your workspace, rerun the command and  and copy here full output of dexcalibur ?

Sure.
https://dpaste.org/pyKJ/raw

Okay, i found the issue.  Is it not relatde to APKtool but it is caused by an issue into Device Manager of Dexcalibur.
Please, can you send me the output of "adb devices" command ?

I close this issue. Feel free to reopen it, if the problem persists with latest version.


🙏Resize时ResizeInfo需要透出resizedWidth和resizedHeight属性

🖋 Description
🏞 What problem does this feature solve
🧐 Suggest the API

name
type
default
description

-
-
-
-

:tada: This issue has been resolved in version @antv/s2-v1.25.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version @antv/s2-react-v1.23.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Get request to unparsed JSON on startup

Problem
Get an error on startup where there is a get request to a file url with an unparsed json object like so:
file:///var/mobile/Containers/Data/Application/<some_long_number>/Library/Application%20Support/<my_application_name>/cordova-hot-code-push-plugin/<build_date>/www/[object Object]
This causes program halt on some android devices.
Steps to reproduce
Install app with hcp. Start up.
System info
Run cordova info (or ionic info) and paste it here, or give us information like:
Cordova CLI: 6.3.0
Gulp version:  CLI version 3.9.0
Gulp local:   Local version 3.9.1
Ionic Framework Version: 2.0.0-beta.11
Ionic CLI Version: 2.0.0-beta.37
Ionic App Lib Version: 2.0.0-beta.20
ios-deploy version: 1.8.6
ios-sim version: 5.0.8
OS: Mac OS X El Capitan
Node Version: v6.2.2
Xcode version: Xcode 7.3.1 Build version 7D1014

This causes program halt on some android devices.

Feels like it's in iOS, not Android.... Can you give a bit more detailed info on how to reproduce this? Step-by-step for the fresh project.

I am very sorry. My fault.
I finally got the time to dive deeper into this issue, and interestingly enough it seems to be triggered by a background geolocation plugin.
Im closing this issue.


Image isn't loaded when electron-builder is used.

Hello,
I have been trying to create a packaged release version of this repo and logo.png isn't being loaded properly (Failed to load resource: net::ERR_FILE_NOT_FOUND) . I think the path to the image is somehow being broken when electron-builder packages the project. The project runs fine when I run it using npm run.
Other people have had the same issue with the boilerplate:
https://github.com/electron-react-boilerplate/electron-react-boilerplate/issues/250#issuecomment-227494838
Have you experienced this issue and if so do you have a workaround?

How did you resolve?


Removed semicolon from rollback stmt for oracle

Fixes #2563 (hopefully all tests will pass) 🙏

Failures seem unrelated 🤔

@atiertant any idea why that broke stuff and if this fix is fine?

@elhigu Interesting, it was failing for 2 platforms when I submitted (node 4 & 8, I think) but everything seems to pass now. It recovered overnight :)

@cemremengu I saw only node 8 failing and reran those tests. They were only random failures, but my question to @atiertant was about this semicolon issue and if its fix is fine (this PR is missing regression test though).

Wanted to add tests but wasnt sure where to add them exactly. It would be great if you give me some pointers.

@elhigu not sure that semicolon is the problem, maybe the problem is that the connection is reused after end of transaction (this could explain the random failures with pool).

@atiertant @elhigu
I apologize maybe I should provide more context here to be more clear and not take your time:
I was using this plugin and it was giving an error on this line. Changing it to ROLLBACK (I submitted a PR) solved the problem.
Now playing around with the knex itself. Looks like trx.rollback() is not producing an error but if I do trx.raw('ROLLBACK;') I get the same error. Although they do rollback similarly (at least from what I understand and why I submitted this PR), maybe it was a bug only in this plugin and not knex itself?

@cemremengu
i think the problem is raw method because it is not redefined in transaction, so it should use client's one and then use a connection from pool not the current transaction one.

@elhigu Is there anything yet to happen with this PR?

This seems to have broken transactions in mysql.

@tgriesser We need more tests :(

I'm also not sure if oracledb / mssql transactions are working correctly looking this code: https://github.com/tgriesser/knex/blob/232fe9f1517dba927f6a3a1fb1b8842d7c0a4007/src/util/batchInsert.js#L51
Also even with hack above batch insert test fails randomly on oracledb.
Transaction implementation in mssql and oracle are quite similar.

@tgriesser We need more tests :(

Sure, we also need to simplify a lot of the internals as well. I'll add a failing case in fixing this.
@elhigu Yeah I don't even think that the oracle driver uses this codepath, they seem to have their own transaction implementation internal to the connection? It's all a little tough to follow, seems a lot of this can be simplified.


fixed issue with reduce_dims and preserve dims kwargs. Added unit tes…

Looks good to me. I re-ran the FlipFlop notebook just to make sure, although the bug and these changes shouldn't have been exercised by the notebook in any case.


Enhanced support in weather-dl for downloading data across month ranges spanning multiple years.

I have a suggestion for an enhancement to the weather-dl tool. Currently, when downloading data for a month range that spans multiple years, the process requires manual splitting of the month range into separate configurations and executing multiple jobs for each part. This process can be tedious and time-consuming.
For instance, if we wish to download data from April 1994 to August 2012, and the target path follows this format: (gs://xyz/{year}/{year}{month:02d}_hres_var.grb2), we need to split the dates into three parts as follows:
1st part :
year=1994
month=04/to/12
day=all
For the 2nd part:
year=1995/to/2011
month=01/to/12
day=all
And for the last part:
year=2012
month=01/to/08
day=all
I propose an enhancement in the weather-dl parser by introducing a new keyword, such as month_year, in the configuration.
The value of this keyword would look like 08-2012 or 01-2022. Additionally, we can add a script to the parser of weather-dl that can correctly interpret this keyword.
P.S.: A similar use-case can be found in the Arco-era5 dataset. Please refer to any file containing sfc in the filename for more context.

I definitely see the value in this language feature, I think it would reduce a lot of boilerplate. Can we explore a few options for what the keyword or syntax to express this? How about you propose a few options, and we'll decide together on the best approach.


GPT-4 model logs not fully appearing in ApiManagementGatewayLogs

This issue is for a: (mark with an x)
- [X] bug report -> please search issues before submitting
- [ ] feature request
- [X] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

When using gpt-4, there are several parameters that are not being logged.
Examples:

OperationId is ChatCompletions_Create not completions_create as lined out in the docs
model, prompttokens, responsetokens etc. columns are not being expanded and are not present in the table because BackendResponseBody contains no data even after many successful queries to APIM.

Example: This KQL query yields no results, just empty data
ApiManagementGatewayLogs
| project BackendResponseBody

How do we log GPT-4 models using this solution?
Minimal steps to reproduce

The formal docs

Any log messages given by the failure

Fails silently

Expected/desired behavior

The same verboseness shown in the log examples would be present with GPT-4 models as well.

Mention any other details that might be useful

Thanks! We'll be in touch soon.

Thoughts on this one?

Hello,
I really need to know if:
a. This solution works with GPT-4 OR
b. If I'm doing something wrong
I have the solution deployed, but again, I don't see the proper logs when queried through KQL.
Thank you

Hello,
Does anyone have any insights on this?

@jakeatmsft, could you advise on this item?
Thank you

Addressed here https://github.com/MicrosoftDocs/architecture-center/issues/4073

Use this query it will work;
ApiManagementGatewayLogs
| where OperationId  == 'ChatCompletions_Create'
| extend model = tostring(parse_json(BackendResponseBody)['model'])
| extend prompttokens = parse_json(parse_json(BackendResponseBody)['usage'])['prompt_tokens']
| extend completiontokens = parse_json(parse_json(BackendResponseBody)['usage'])['completion_tokens']
| extend total_tokens = parse_json(parse_json(BackendResponseBody)['usage'])['total_tokens']


Make StreamField values mutable and deprecate stream_data

Fixes #2876 and supersedes #2886. StreamValue now implements the MutableSequence interface so that it can be modified in place, rather than having to reassign the entire list to update it:
# Replace the first block with a new block of type 'heading'
my_page.body[0] = ('heading', "My story")

# Delete the last block
del my_page.body[-1]

# Append a block to the stream
my_page.body.append(('paragraph', "
And they all lived happily ever after.
"))

# Save the updated data back to the database
my_page.save()

StreamValue has undergone some internal refactoring to make this possible: previously the internal stream_data property contained the underlying data in one of two possible formats, depending on whether the data came from the database ('lazy' mode, where it's stored as JSONish dicts and converted to native Python values on first access) or from Python code ('non-lazy' mode, where it's stored in tuples already containing the native Python values). This setup doesn't work for mutable sequences (where we might have some data items originating from the database, and others overwritten in python code) and is pretty messy anyhow.
Unfortunately, stream_data has found its way into a lot of end-user code that doesn't properly account for the lazy versus non-lazy cases (which usually means it mostly works, but breaks on page previews) and so this needs to go through the deprecation process. I've added a backwards-compatible shim for stream_data, along with a new accessor raw_data which consistently returns the JSONish data (i.e. it does what people think stream_data did, and for 99% of cases people should be able to fix their code by replacing stream_data with raw_data).
Note: I've written the deprecation warnings and docs as if this is a 2.11 feature, but I'm not expecting it to make the deadline :-) Will update these to target 2.12 once the necessary bits of release housekeeping have been done. (done)

Now rebased on current master to target 2.12.

Merged in 93a80efe5061e5f0f31b7eb8fb2b36dc83cd3e6c with release notes added in 8d9e864581e1772885ace7efbf9c7253bf989a93


Analyzer fails to resolve implicit .call getter on nullable type via extension

The following code produces an error in the analyzer, but no error in the CFE:
class C {}
extension on C? {
  void Function() get call => () {}; // (1)
}
f(C? c) {
  c(); // ERROR: The expression doesn't evaluate to a function, so it can't be invoked.
}

I believe the CFE behavior is correct.  c() should be considered equivalent to c.call(), and since call is a getter defined on type C?, it should be allowed.
Note that if line (1) is changed to void call() {} (which should be equivalent), then both the analyzer and CFE agree that there is no error.

I believe the analyzer behavior is correct here, and the CFE is missing an error.  See sections 13.5 and 17.15.5 here.

@leafpetersen oops, you are right.  Here's the relevant text from section 13.5 (The call Member of an Extension) of the spec:

In particular, i' cannot be an invocation of an extension getter whose return type is a function type, Function, or dynamic.


Add extra option to transfer blobs

Add extra option to transfer blobs, and be less dependent on the connection pool sizing.
This option will be more used to be able to transfer a big set of data between databases. E.g. during migrations or upgrades.

@wimmeerschman this setting is only going to be used from your custom code and not by bulkloader code? It may make more sense to create a generic property (e.g. a dictionary) where you can push any setting you want. Adding this setting without usages in this repo makes little sense imo.


Add Indicator to documents telling if they are a url, file or text

I've understood the requirements and would like to work on this issue/feature.


Reinstate instance type toggle

Fix https://github.com/developmentseed/pearl-frontend/issues/37#issuecomment-1543752039.
@geohacker ready for review.

@geohacker I'm merging this, as explained at https://github.com/developmentseed/pearl-frontend/pull/44#issuecomment-1549428135.


OCM-9294 | test: automate id:53570 create cluster with IAM role path

The testing is passed, it covers classic and hcp clusters.
Log: https://privatebin.corp.redhat.com/?64d6b14acbd8cd38#6Mu8DdeKQ9G3Hni5WNc3UoThxzjSimA4p3fP6YcR64hA

/lgtm


simple retry for fetch

Perhaps can update the code in helpers.py, to use aiohttp and asyncio

I can help with this, though I'm not sure how it would work out with an asyncio API. Wouldn't async propagate to the caller?
Otherwise, do you want a user param with a default 1,2, or more? or a decorator if you have other net request functions that it can be applied to?
How about exponential backoff or is it getting unnecessarily complicated and moving away from being tiny? I'm assuming most of the links would be in cloud storage/APIs. If you feel net congestion is a probable case I can add a delay.

@Qazalin this is not a good first issue. On the surface it sounds like one because it's simple, but read the replies and see why it isn't.
A good first issue is something where there's one way to do it and it's easy for a person to tell themselves if they did it correctly.

An example implementation is at https://github.com/Thinklab-SJTU/pygmtools/pull/73/files based on

https://likegeeks.com/downloading-files-using-python/
https://docs.aiohttp.org/en/latest/http_request_lifecycle.html
https://codereview.stackexchange.com/questions/259112/async-download-of-files
https://docs.aiohttp.org/en/stable/client_quickstart.html

I have a solution ready for this with the following capabilities, let me know if it suits you. It helps when these questions are answered before submitting useless PRs.

param 'retries' with default value = 1 and internal constraint of max 4 retries
retry based on http code list: 408, 425, 429, 500, 502, 503, 504.
simple expo backoff with a multiplier of 2
No asyncio.
No decorators


Link from GBIF dataset metadata should correctly filter observations

Belatedly making an issue after an internal conversation related to the text we display associated with our GBIF dataset. The issue: clicking the link that is supposed to display only the observations included in the GBIF export instead includes all research-grade observations because the created_d2 and license parameters aren't working as passed.
As identified by @kueda, three things are needed to make the link work as expected:

we need Explore and the API to support created_d2 alone (without a created_d1) in a url with or without a full ISO 8601 datetime
Explore needs to support the license param (not just the API)
update the GBIF DwC-A metadata so the URL uses lowercase license codes instead of uppercase.

For 1), if I use the filters to set a created_d2 date, it filters appropriately. But if I take the link that generates, and paste it into a new tab, it doesn't filter by created_d2. Here's an example of my RG observations created before Jan 1, 2016: https://www.inaturalist.org/observations?created_d2=2016-01-01&place_id=any&quality_grade=research&subview=grid&user_id=carrieseltzer (it should display 570, but instead displays all 6,824). Clicking into filters shows 2016-01-01 already in the "date added" range, and selecting the "range" radiobutton then brings the right results.

#2874 is also related to point 1.

Since #2874 is closed, only these two items remain to properly filter the observations from the GBIF dataset metadata:

[ ] Explore needs to support the license param (not just the API)

[ ] update the GBIF DwC-A metadata so the URL uses lowercase license codes instead of uppercase.

Since #2874 is closed, only these two items remain to properly filter the observations from the GBIF dataset metadata:

[ ] Explore needs to support the license param (not just the API)

[ ] update the GBIF DwC-A metadata so the URL uses lowercase license codes instead of uppercase.

license seems to have some unexpected case-sensitivity.  If you choose from the Photo Licensing drop-down in Explore's More Filters, the URL populates with a capitalized photo license, e.g. https://www.inaturalist.org/observations?photo_license=CC0&place_id=any.  However, it will also accept lowercase, e.g. https://www.inaturalist.org/observations?photo_license=cc0&place_id=any.  The behavior for license is not like photo_license -- it will only accept the lowercase version, e.g. https://www.inaturalist.org/observations?license=cc0&place_id=any.

license seems to have some unexpected case-sensitivity.  If you choose from the Photo Licensing drop-down in Explore's More Filters, the URL populates with a capitalized photo license, e.g. https://www.inaturalist.org/observations?photo_license=CC0&place_id=any.  However, it will also accept lowercase, e.g. https://www.inaturalist.org/observations?photo_license=cc0&place_id=any.  The behavior for license is not like photo_license -- it will only accept the lowercase version, e.g. https://www.inaturalist.org/observations?license=cc0&place_id=any.

As far as I can tell the URL is now working, as is the link to Carrie's RG observations created before Jan 1, 2016.

As far as I can tell the URL is now working, as is the link to Carrie's RG observations created before Jan 1, 2016.


Fixed link for Spock docs

The old link goes to a 404 page.

Thanks again! :+1:

Cheers!
On Tue, Feb 9, 2016 at 8:04 PM, Nadav<EMAIL_ADDRESS>wrote:

Merged #35 https://github.com/nadavc/groovykoans/pull/35.
—
Reply to this email directly or view it on GitHub
https://github.com/nadavc/groovykoans/pull/35#event-544700557.


Declarative Pipeline - Support CustomPipelineTemplate

Overview
The terraform-pipeline supports setting up a Declarative pipeline. Declarative pipelines give us the advantage of restarting pipeline from a stage from a previous run. "Restart from Stage" requires the stash artifact to be available to use in a subsequent stage.
This issue can be resolved by using a CustomPipelineTemplate. However, Jenkins throws an error when we use a CustomPipelineTemplate.
Version
Pipeline 2.6 version on Jenkins
terraform-pipeline 5.6
Error
groovy.lang.MissingPropertyException: No such property: none for class: CustomPipelineTemplate
<EMAIL_ADDRESS>'terraform-pipeline-customizations']) _
Jenkinsfile.init(this)
Jenkinsfile.declarative = true
Jenkinsfile.pipelineTemplate = this.CustomPipelineTemplate

vars/CustomPipelineTemplate.groovy
pipeline {
     agent none

     options { preserveStashes() }
     stages {
         stage('Validate') {
             steps {
                 script {
                     validate.build()
                 }
             }
         }

         stage('Build Artifact') {
             steps {
                 script {
                     buildArtifact.build()
                 }
             }
         }
  }

Per the README.md, vars/CustomPipelineTemplate.groovy should define a method call(stages) and then you use ((Stage)stages.getAt(index)).build().

I tried this but it didnt work. The pipeline gives an error saying
groovy.lang.MissingPropertyException: No such property: any for class: CustomPipelineTemplate.

Talked with sujikann, and confirmed the problem.

Follow the README and create a custom template:

// var/CustomPipelineTemplate.groovy
def call(args) {
    pipeline {
        agent none

        stages {
...

Create a declarative pipeline, and use your template:

Jenkinsfile.init(this)
Jenkinsfile.declarative = true
Jenkinsfile.pipelineTemplate = this.CustomPipelineTemplate

Expected: you get a declarative pipeline using the code provided by your template
Actual: you get the error groovy.lang.MissingPropertyException: No such property: any for class: CustomPipelineTemplate.

Looks like there's a defect in the README itself.  For some reason, naming your variable stages causes some sort of problem.  If you rename the variable to args, the problem goes away.
Example:
// var/CustomPipelineTemplate.groovy
def call(args) {
    pipeline {
        agent none

        stages {
...

The docs should be fixed, so that if you copy/paste the example code, it works.

Looks like there's a defect in the README itself. For some reason, naming your variable stages causes some sort of problem. If you rename the variable to args, the problem goes away.
Example:
// var/CustomPipelineTemplate.groovy
def call(args) {
    pipeline {
        agent none

        stages {
...

The docs should be fixed, so that if you copy/paste the example code, it works.

Gonna reopen the issue, only because the documentation is incorrect and should be fixed, to prevent this problem/confusion in the future.


pro new 的项目报以下错误

Module build failed: TypeError: Property right of AssignmentExpression expected node to be of a type ["Expression"]
but instead got null
at Array.map

Property right of AssignmentExpression expected node to be of a type ["Expression"]
报错
Property right of AssignmentExpression expected node to be of a type ["Expression"] but instead got null

解决
删除 .webpackrc.js 里的 transform-decorators-legacy。
参考
https://github.com/ant-design/ant-design-pro/pull/1345


introduce dotnet6

a new runtime dotnet6 is now available on AWS

source: https://aws.amazon.com/jp/blogs/developer/net-6-on-aws/


ln command v0.0.1

add ln command

When adding new commands, please also update the README.md file, to increment the number of completed commands, and put a checkmark next to the one you did.

A weird error when compiling for Windows... we'll get that sorted out.


Add support for US Gov cloud in porter azure plugin

Description
As a Azure TRE Developer
I want the porter azure plugin to support vaultUrl
So that I can use KV which are not on public cloud.
Link to issue in the plugin repo: https://github.com/getporter/azure-plugins/issues/57
Technical notes:

Patch solution for reference: https://github.com/getporter/azure-plugins/compare/main...LizaShak:azure-plugins:main
I understood their approach is to pass vaultUrl as additional param, and use it if it is passed. Please verify with Caroline.
See Developer Tasks in : https://github.com/getporter/porter/blob/main/CONTRIBUTING.md to see how to build the plugin and test it.

Acceptance criteria

[ ]  A fixing PR is merged on the repo: https://github.com/getporter/azure-plugins
[ ]   azure plugin is able to connect and get secrets from KeyVault in US Gov cloud
[ ]   azure plugin is able to connect and get secrets from KeyVault in public cloud

https://github.com/getporter/azure-plugins/pull/60
Waiting for Carolyn's review


Add basic cli tests using Bats

So far just a few tests to run gcalcli with no commands and with --help, but it's a foundation we can build on incrementally.
Fixes #419.

Now to see if the new tests actually run successfully under GitHub Actions...


Increment versions for mysqlflexibleserver releases

Increment package versions for mysqlflexibleserver releases

API change check
API changes are not detected in this pull request.


Exception 0xC0000225 on v3.0.8535.4008 Git-1048ad22 after start on Win7 SP1 x64

Exception 0xC0000225 on v3.0.8535.4008 Git-1048ad22 after start on Win7 SP1 x64ю Decode dump screensot:

Exception 0xC0000225 on v3.0.8535.4008

Crashes can be expected when you build the source. That version doesn't exist and hasn't been released 🤷♂

Git 5112a29e work w/o problems. Big thanks!


When I try to create records that have a single digit record selector field I get an error

Is there an existing issue for this?

[X] I have searched the existing issues

Version of Corteza
No response
Current Behavior
Unable to create records associated to other tables/modules when the other table has a single digit id. The field type is a number but because the id is < 10 we receive the following error:
Error: Could not create record: invalid reference format
I think I’ve tracked this error down to the RecordValueSanitization function in the record.go file.
It looks like it’s checking for a numeric using the regex ^1-9$ which fails for single digit numbers.
There is a Numeric regex in patterns.go that works for single digit numbers in addition to anything >= 10
Numeric string = “^[0-9]+$”
Expected Behavior
Should be able to create records that have a field, type Record selector, with a single digit value, value < 10.
Steps To Reproduce
Create a module connected to an external table that has at least one record where the record ID < 10.
Create a second module that has a field that is a record selector type where the Module name is set to the module above.
Create a record in the second module and populate that record selector field with a record from the first where the ID < 10.
Click save and receive error
Environment and versions
Corteza version: 2023.3.4

Anything else?
Link to discussion in forum: https://forum.cortezaproject.org/t/record-create-failure-for-single-digit-record-ids/1548

Have you checked if changing that condition brings implications to anything else? I don't think there is anything wrong with changing those constraints, but I'm a bit spooked at what else would crash so a bunch of testing would need to be done.
IIRC there used to be a condition (somewhere) that all the IDs had to be of length x to further limit it to sonyflake unit values (but I could be wrong so don't quote me on it)
There were some ideas/thoughts we should support different identifier types within DAL for when you connect to external sources, but nothing concrete as of now.

As a workaround
As a workaround you could consider altering your DB rows to add x to all IDs; for example, 10 so the condition would pass with no questions and minimal issues (since all IDs would just be displaced by x); is that an option for you until we do some proper fix?

Manually adding 10 to all IDs in all tables we would connect to is not a viable option for us. These tables have a fair amount of relationships to other tables and since we are talking about the ID field itself, which would be the foreign key in other tables, that would not be a road we would prefer to go down.
I understand sonyflake values and them being used for the ID field in Corteza created tables but this error presents itself with other/external/non-Corteza created tables. This particular piece of code wants a number in the ID field and single digit IDs are failing due to this issue.
I didn't see any comments in the code, where this is checked, that suggests the original intent for this specific check was for single digit numbers to fail the 'IsNumeric' check. Any/all double digit numbers pass, like 10, but a double digit number isn't a valid sonyflake ID so I'm not sure that sonyflake requirements have anything to do with this particular code/check.

Thought so.
Probably just a lapsus on our part but it'll still need a bunch of testing

Is there any other information needed for this issue to be worked?

@KinyaElGrande looks like it'll most likely be a regex issue (see + instead of * in te first search result).

Please do check if this is the only thing that needs to change; think this should be it but I'm not entirely sure

Is there some workaround for this besides us altering our DB rows to add x to all IDs and making the same change on all tables that use that ID as a foreign key?
This stops the creation of records associated to other tables/modules when the other table has a single digit id. There are plenty of options for using external tables with Corteza which would be impacted by this issue.

There is a Numeric regex in patterns.go that works for single digit numbers in addition to anything >= 10
Numeric string = “^[0-9]+$”

Hi @m4thfr34k and @dcharpen the issue has now been resolved with https://github.com/cortezaproject/corteza/pull/1614.
You are now able to reference single-digit record IDs using record selector without any error.


A great idea

Hi, i love vscode
When i create an js module ,
step 1: need to open the context menu
step 2: need to create a folder
step 3: create an index.js or index.ts  file
Now, Will we to offer an option "create module" for the context menu that can create a folder with an index.ts file? so i just to fill a name of the folder.
Next, some programe language has itself module.
As i know, rust mod default file is mod.rs rust lib default file is lib.rs
Can we offer an api to LSP, so extendsion can set the create module way by itself?

Seems worth exploring with an extension first. If enough extensions/languages need this sort of thing, we could consider it for VS Code / LSP
Also please give issues a descriptive title

Seems worth exploring with an extension first. If enough extensions/languages need this sort of thing, we could consider it for VS Code / LSP
Also please give issues a descriptive title