Extend and rework ImportSQL.ps1

This change extends the ImportSQL.ps1 script, so that it automatically grants read permissions to the new cert. If the SQL server has no read permission for the private key, it will fail to run.
It also restructures the script a bit for easier debugging and extends the help.

Hi @merlinschumacher! Thanks for this contribution, it looks like a good upgrade for the script. But I won't be merging any more PRs in this repo, because I'm working on a new cross-platform version of the program called simple-acme instead. Would you please consider contributing this to https://github.com/simple-acme/simple-acme/?


Recommendations and sample code for pdf

updated the code with two more functions: 1. execution time for each function 2. adding progress bar for image-based pdf files (the extraction of image-based pdf takes much longer; it's nice to have a progress bar to see if things are going well)


Add documentation for simulating PayID payments

What
Add documentation on how to test incoming PayID payments in Sandbox.

Add a new Sandbox Only section to the sidebar.
Add endpoint documentation for simulate/incoming_payid_payment.
Add a link to the new endpoint documentation from the existing add receivable contact section.

LGTM.  Do we want to add any character lengths here too?  Only because Ladbrokes have been pushing it real hard.
Foe example, on the Add a receivable Contact docs


Is aarch64 or arm64 supported?   OCP has those images.

Version
$ oc-mirror version
Client Version: version.Info{Major:"", Minor:"", GitVersion:"4.11.0-202208020235.p0.g3c1c80c.assembly.stream-3c1c80c", GitCommit:"3c1c80ca6a5a22b5826c88897e7a9e5acd7c1a96", GitTreeState:"clean", BuildDate:"2022-08-02T04:04:36Z", GoVersion:"go1.18.4", Compiler:"gc", Platform:"linux/amd64"}

What happened?
Seems aarch64 or arm64 do not work as architecture types?
What did you expect to happen?
Pull down arm64 or aarch64 images for openshift.
How to reproduce it (as minimally and precisely as possible)?

$ cat architecture-imageset-configuration.yaml
apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
mirror:
  platform:
    architectures:
      - "arm64"
    channels:
      - name: stable-4.11
$ oc-mirror --config=architecture-imageset-configuration.yaml docker://provisioning.schmaustech.com:5000 --dest-skip-tls --dry-run
Checking push permissions for provisioning.schmaustech.com:5000
error: invalid configuration: release architecture "arm64" is not a supported architecture

Anything else we need to know?
Enter text here.
References

Enter text here.

@schmaustech Thanks for pointing this out. There was an extra layer checking the release architectures (which did not have an all-inclusive list) in the config that I removed in a PR to let the Cincinnati APs determine valid architecture submissions.


nginx pod fails after microk8s update t 1.22.3+

the nginx pod is no longer running after an update to 1.22.3+. any idea how to fix this?

made a fix by updating image of nginx daemonsets to point to this version k8s.gcr.io/ingress-nginx/controller:v1.0.0@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6  and the error dissappear

Hi @frendcarlo
Could you provide some more info as to how/why the nginx controller stopped working on v1.22.3? I assume you are doing your own builds or 1.22.3-rc0 ?

I cannot confirm this issue. The ingress controller on 1.22.3-rc.0 looks fine to me.

I made some changes on daemonsets nginx-ingress-microk8s-controller to add extra ports that i needed, but the container image is pointing to k8s.gcr.io/ingress-nginx/controller:v0.35.0, and after the update, this points to k8s.gcr.io/ingress-nginx/controller:v0.49.1 which causes the nginx pod fails to run because of this error  > Failed to list *v1beta1.Ingress: the server could not find the requested resource.
I change this to v1.0.0 and it works fine now.

Oh i see,. When we upgrade microk8s, we don't upgrade addons previously enabled.  It has to be manually disabled/enabled.  For stateless services this is easy.
Moving from one version of kubernetes to another usually introduce breaking changes.  As such we recommend you use a specific channel
From v1.22 we upgraded nginx to the one that is compatible.  I think calico also needs some upgrade.

i see, i will check calico, thanks for the info.

Just chiming in here, I think I also hit this recently after an auto upgrade.  Did not have time to debug, but switched back to 1.22 and it was resolved.  I did not realize the addon disable/enable upgrade workflow was necessary but I see that in the docs now!


Apply Black formatting to code base

Hey @enty8080 , I've added the Black formatter to the CI pipeline and have applied it (with single quotes) to the entire HatSploit codebase.
Review the format and let me know what you think. I've been using Black for about 6 months and I've been slowly incorporating it where I can. It might help with future contributions, as it standardizes code formatting.

@naltun Hey, I can not merge it due to checks fail. Please fix everything and I'll merge it.
Looks like black command not found.


Add native flags

Since we are building on the same hardware that is running, we can take full advantage of the compiler's knowledge of the CPU. I have succesfully built OpenCV adding the flags:
  -D CMAKE_CXX_FLAGS="-march=native -mtune=native" \
  -D CMAKE_C_FLAGS="-march=native -mtune=native"

I haven't benchmarked it, but on my experience with more complex CPUs, that usually gives me a 5-20% improvement.
Should I open a PR?
BTW: thanks for the scripts! It is great!

Thanks!
If you like to open a PR, be my guest.
Otherwise I will take care of it.


Funds show only two first currency

Hi!
The menu displays only two Funds currency. Prompt where to display all correct me added to currency.yml.

+1

In my case, the problem was suddenly solved. The last thing I did before that is: make a deposit in my added fiat currency and confirm the deposit.

Question: did you have any issues getting the main trading board to quote bitcoin pricing/stream the pricing and what files seemed to make sure it worked. That's problem I'm having and hoping for help if possible

so how to solve this issue? I have same issue on funds page

Yes, solve, add currency number to DB table contains user balance!

what does that mean? Can you tell me that step by step? I am new to peatio:)


cleanup spaces

found some weird spacing while reading the code

Hi @mrluc , my OCD is killing me :) Could you kindly take a look at this please? Thank you!


Error reported in CADENCE 17.4

At the command line, enter exportjson and enter error when prompted。
exportjson
Exporting .json for variant: EUROPE
Error pcreReplace: argument #2 should be a string (type template = "ottxx") - t

Is it possible for you to share the design? Unfortunetly debugging in the skill environment is not as easy as one might think ...

demo.zip
This is my test project.

您可以分享设计吗？不幸的是，在技能环境中调试并不像人们想象的那么容易......

Thank you for sharing! I can reproduce the error and look into it as soon as possible!

谢谢你的分享！我可以重现错误并尽快调查！

Thank you.

Check out my last commit and let me know, if it is working for you.
I could find the reason for the error and made a quick fix, which should solve your problem, but I'm not sure, if there is a little bit more to do ...

Check out my last commit and let me know, if it is working for you. I could find the reason for the error and made a quick fix, which should solve your problem, but I'm not sure, if there is a little bit more to do ...
I used the new version and still the same error. Here are the statements I added in allegro.ilinit。

This is lastly error.
WARNING No variant.lst found, exporting entire design ...
Loading skillExt.cxt
Error pcreReplace: argument #2 should be a string (type template = "ottxx") - t

I deleted the previous file and now it works. Thank you

In allegro you have to be careful: deleting a file does not affect the code execution immediately, but only after the next restart. If you want to see an effect directly, you need to reload the file by tying load( filename ).

Found a new issue, I've successfully exported json, but conversion fails with InteractiveHtmlBom.

test.zip

Did you downloaded the newest version of the InteractiveHtmlBom?
I think this is a bug, which should be fixed now.

Did you downloaded the newest version of the InteractiveHtmlBom?
I think this is a bug, which should be fixed now.

Thanks. I used the last InteractiveHtmlBom,it is successed.


Clean Up Dependency Injection

Our current "poor man's dependency injection" approach isn't going to cut it much longer, we should rework it to better manage resources.

I'd like to take a stab at this. Does Autofac work for you?

Yes, Autofac is good, and let me know if it gets tricky.

We've added autofac, but we still need to clean up here.

@benrick Next steps I'm going to start on tonight are primarily two things,

Inspect all the interfaces #107 created, find our common functionality, and pair it down if possible, then
Break down modules (games, non "core" commands, twitch, etc). I'd very much like to have the only type references in the DevChatter.Bot project to be from DevChatter.Bot.Core, while everything else must implement through a single entry point per type of module (one for commands, one for chat clients).


Mod Ideas

[ ] Extended Pilgrimage: adds another pilgrimage after the Seven Graces
[x] KCD-style pickpocketing
[ ] Simple levelling mod: just remember the skill increases from last time
[ ] Lucky day mod: each day your luck gets randomised, but can't sink below your luck attribute

[ ] Immersive Travel mod: basically abots mod but without stutter and in lua

[ ] Disco Elysium Thoughts for MW: a simple interpretation of the thoughts system, after completing certain quests you form a thought - which you can internalise or discard and which adds some buffs or debuffs

Ideas
UI/HUD

[x] Loading Splash Screens (https://www.nexusmods.com/morrowind/mods/51076)

Displays Splash screens when a (cell) loading screen happens
Screens are taken at random from the available splashes

[x] Compare Tooltips (https://www.nexusmods.com/morrowind/mods/51087)

Compare tooltips for item
TODO: display two tooltips at once next to each other

[x] Immersive Realistic Maps

Disable the worldmap and overlay (optional) OR
When pressing M open a contextually-chosen map depending on player inventory

If a "Guide to X" is in the inventory, open the book
If a "Map bundle" is in the inventory, open the map

Crafting Framework: Add craftable maps from the books
Crafting Framework: Add craftable "map bundle" from books/maps
Tooltips when hovering over a city (taken from the vanilla books)

[ ] Barbers

Add "Barber" dialog to publicans? -> Opens the/some character creation screen to switch haircuts in-game

Gameplay

[ ] MULE vanilla level-up screen add-on

??? make MULE only level up when sleeping
re-add the vanilla level-up screen

[ ] Better Vanilla leveling

Skills give level up points normally
optionally let minor skills contribute too
on level-up and sleeping:

display the level-up screen, but
disable player choice for attributes
all skill progression contributes to an attribute level up
handle luck somehow

Balance this ?

[ ] experience based levelling

Replace skill levelling with XP-based levelling
Quests give XP
Kills give XP (optional)
Skill levelling givs XP

Lore/Immersion

[x] Bosmer Green Pact (https://www.nexusmods.com/morrowind/mods/51123)

change dispose of corpse to "eat corpse"
TODO: Ashfall integration
TODO: Hunting integration
TODO: better buffs
TODO: debuffs
TODO: better class integration

[ ] Immersive Rotten Corpses

Change the name of corpses depending on how long they are dead
Change the mesh depending on time of death
VFX

[ ] Ashfall hunting and skinning

Field dressing of corpses (skinnnig)
Gutting/butchering of corpses (logic: make food quality/quantity dependent on field dressing, skill, temperature)
Add rotting (dep on temp, field dressing)


Nothing printed in logs when using .json()

I'm using the method call Logger.t("tag", 0).json(jsonString) but nothing prints in the logcat.
In case it helps I'm using the following JSON
{"coord":{"lon":-121.49,"lat":38.58},"sys":{"message":0.007,"country":"United States of America","sunrise":1432039821,"sunset":1432091674},"weather":[{"id":803,"main":"Clouds","description":"broken clouds","icon":"04n"}],"base":"stations","main":{"temp":279.79,"temp_min":279.79,"temp_max":279.79,"pressure":1018.82,"sea_level":1031.88,"grnd_level":1018.82,"humidity":91},"wind":{"speed":1.51,"deg":167.501},"clouds":{"all":64},"dt":1432033881,"id":5389489,"name":"","cod":200}

Not a bug.
I had created a log filter with "Info" set as the minimum level so I didn't show.


Added missing props to extension for dimension in json-stat2

Added:

timeUnit
measuringType
priceType
adjustment
baseperiod

Should we change to refPeriod and basePeriod?


Simplified tests directory structure a bit

what about the namespaces? this will also affect AR tests in extensions as they copy some classes from here.

what about the namespaces?

What do you mean? They are not affected anyhow.

this will also affect AR tests in extensions as they copy some classes from here.

yes, those tests should be adjusted later.

Migrated to #8102


Exporting "#" in hashtags.

Debug log ID
YTH373FG-euc/6.7.162-6
What happened?
Using the combination of
"zotero-storage-scanner"
"zotero-better-bibtex" and
lualatex+biber, I have the following problem:
Zotero-storage-scanner creates "#nosource"-tags, which are then exported by zotero-better-bibtex as keywords (with the "#") into the .bib-files. These crash then in the longtem the combination of lualatex+biber, since it does not expect the hashtag in the keyword-field.
I'm not sure, where excatly the "error" is, but the combination is fatal.

 The ID will have -refs- in it.

@njbart, between this and #2781 I don't know what to make of keywords fields. Escaping _ breaks tag filtering, but # can't be in the keywords field without escape.

Like that?
NZZVUN8A-refs-euc/6.7.162-6
That's one entry, which has #nosource as tag (the nosource without hashtag came because I renamed all tags at another computer to fix the problem temporarily).

 It's apparently not simply verbatim-or-not.

Looks like tags tolerate everything except # and %.

I think it's more likely that people will want to filter on tags than want to print them, and since  escaping breaks filtering, I'm going to keep that as the primary use case. Printing will still be possible with the workarounds you mention.
I have to admit - I don't know what "literal mode" entails. Is that what I might know as regular document body mode?

I think it's more likely that people will want to filter on tags than want to print them, and since escaping breaks filtering, I'm going to keep that as the primary use case. Printing will still be possible with the workarounds you mention.

Agree.

I have to admit - I don't know what "literal mode" entails. Is that what I might know as regular document body mode?

Yes. "[LaTeX] regular document body mode" captures it quite nicely.

As an aside -- is there a difference between -- and \textendash? This seems to indicate that \textendash should be preferred and that -- requires extra packages/config, but I've always used -- without having to do anything.

Extra packages: None required.
Whether you're using the one or the other might have effects on hyphenation though, and it seems if you do want to allow a linebreak you should use -- (see here).


Enhancements for ember 4

The Ember framework has evolved substantially since we first constructed the ember-data-table.  When this component was envisioned, Data Down Actions Up (DDAU) was not a real thing yet and variables were passed with two-way binding.  We were discovering something like named blocks would be handy but we did not have solutions.
We now have tracked properties, DDAU, named blocks and Ember is moving towards being more explicit about integrations.
It has been clear for a while that the ember-data-table could be upgraded to cope with this new reality.  We have also noticed that we'd prefer to expose more basic building blocks so alternative strategies can be implemented (like cards).  This PR works towards upgrading the ember-data-table to modern Ember with named blocks.  It also gets rid of the mixins that are currently in place.  This has been a fairly mechanical effort up to this point and improvements can be made.  Some things we see:

the code likely has bugs due to mechanical replacement
naming of the components can be cleaned up (the named block and the component name are not shared)
some libraries can be stripped, as suggested in #30
arguments (and their names) provided through the tree can likely be improved
for docs, the RawDataTable component could serve as an example to make data-table flavours use the same API (but upgrades will be manual)

A PoC bootstrap ember-data-table has been built based on the included raw-data-table.hbs file.  It would be wise to have a few more of these cases so we can see if other arguments would need to be added.

Could this be released as a beta so it's easier to experiment with in our apps?

Yes, no, maybe, sure, not yet but maybe (*)?  I think it makes most sense to try converting a few things with everything linked so we can update the renderless components at the same time.  First tests show there's some streamlining of variables we should execute (these used to be internal state but are now much more a public API), and we expect some bugs to be in here.
I'm good with setting up a build for this if you'd want to gamble and run it somewhere, but I'm even more game for hacking on your use-case and seeing if it fits and where the friction is.  I'm fairly certain the API is not stable as it stands, though the concept would stay the same and can easily be evaluated.  If the concept stands, most changes will be name changes.
This is an open response, any thoughts?  Go for a build anyways?  Go yolo and base ourselves on a git tag?

Ah I think I misunderstood how mature this was, perhaps a release is a bit early in that case.
FYI: From experience ember addons often have issues when installed from a git tag because npm tends to run the install scripts a bit different when doing that (I believe the prepare or prepublish step isn't ran or something like that, I'm sure there are good reasons for that but :exploding_head: ). That may or may not be an issue with this addon.


Activating the "MicrosoftMLModelBuilderAutoML" service failed

System Information:

Model Builder Version (available in Manage Extensions dialog): v<IP_ADDRESS>3003
Visual Studio Version Microsoft Visual Studio Professional 2022 (ARM 64-bit) - Preview, Version 17.12.0 Preview 5.0

Describe the bug

On which step of the process did you run into an issue: After choosing a CVS/Text file to analyze
Clear description of the problem: The Model Builder wizard fails on the Add Data step

To Reproduce
Steps to reproduce the behavior:

Select Add Machine Learning model in a context menu
Choose Value Prediction option
Go to the Data step
Choose any CVS or Text file
See Model Builder Error dialog

Expected behavior
The data is imported and analyzed, Data Preview is displayed
Screenshots

Additional context
   at Microsoft.ServiceHub.Framework.RemoteServiceBroker.<GetProxyAsync>d__361.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Threading.Tasks.ValueTask1.get_Result()    at Microsoft.VisualStudio.Utilities.ServiceBroker.GlobalBrokeredServiceContainer.ProfferedRemoteServiceBroker.<GetProxyAsync>d__191.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Threading.Tasks.ValueTask1.get_Result()    at Microsoft.VisualStudio.Utilities.ServiceBroker.GlobalBrokeredServiceContainer.View.<GetProxyAsync>d__191.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ML.ModelBuilder.AutoMLServiceFactory.d__0.MoveNext() in //src/Microsoft.ML.ModelBuilder/AutoMLServiceFactory.cs:line 68
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ML.ModelBuilder.Actions.DataTextActions.d__14.MoveNext() in //src/Microsoft.ML.ModelBuilder/Actions/DataTextActions.cs:line 275
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ML.ModelBuilder.Actions.DataTextActions.d__2.MoveNext() in //src/Microsoft.ML.ModelBuilder/Actions/DataTextActions.cs:line 61
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ML.ModelBuilder.ToolWindows.DataTextViewModel.d__136.MoveNext() in //src/Microsoft.ML.ModelBuilder/ViewModels/DataTextViewModel.cs:line 683
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ML.ModelBuilder.ToolWindows.TextDataControl.<<BrowseFileButton_Click>b__3_0>d.MoveNext() in /_/src/Microsoft.ML.ModelBuilder/ToolWindows/DataTabComponents/TextDataControl.xaml.cs:line 69`

@dieron Are you running on an arm machine?

Yes. It's a Microsoft ARM Dev Kit machine.

Thanks @dieron  Although I feel like the service activation would still fail, would you might try the latest model builder v17.19.2 from vS marketplace when you have time.

Confirmed. The latest model builder from the VS marketplace fails.

Any update on this issue? I am having the similar problem while trying to import csv file on a macbook running windows with parallels desktop.

@faikozcan We are actively looking that

@dieron and @faikozcan We push a fix for support arm64. Here's the vsix artifact, would you please help us  a favor and see if it works on your arm device?

https://drive.google.com/file/d/1nsQnyrt6pH0zjxR1g_ohkM55zzD1A1pT/view?usp=sharing

Hi
It still doesn't work on my machine. However, I updated Windows recently
Activating the "MicrosoftMLModelBuilderAutoML" service failed.
at
Microsoft.ServiceHub.Framework.RemoteServiceBroker.d__361.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Threading.Tasks.ValueTask1.get_Result()
at
Microsoft.VisualStudio.Utilities.ServiceBroker.GlobalBrokeredServiceContainer.ProfferedRemoteServiceBroker.d__191.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Threading.Tasks.ValueTask1.get_Result()
at
Microsoft.VisualStudio.Utilities.ServiceBroker.GlobalBrokeredServiceContainer.View.d__19`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task)
at
Microsoft.ML.ModelBuilder.AutoMLServiceFactory.d__0.MoveNext()
in //src/Microsoft.ML.ModelBuilder/AutoMLServiceFactory.cs:line 68
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task)
at
Microsoft.ML.ModelBuilder.Actions.DataTextActions.d__14.MoveNext()
in //src/Microsoft.ML.ModelBuilder/Actions/DataTextActions.cs:line 275
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task)
at
Microsoft.ML.ModelBuilder.Actions.DataTextActions.d__2.MoveNext()
in //src/Microsoft.ML.ModelBuilder/Actions/DataTextActions.cs:line 61
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task)
at
Microsoft.ML.ModelBuilder.ToolWindows.DataTextViewModel.d__136.MoveNext()
in //src/Microsoft.ML.ModelBuilder/ViewModels/DataTextViewModel.cs:line 683
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task)
at
Microsoft.ML.ModelBuilder.ToolWindows.TextDataControl.<<BrowseFileButton_Click>b__3_0>d.MoveNext()
in
/_/src/Microsoft.ML.ModelBuilder/ToolWindows/DataTabComponents/TextDataControl.xaml.cs:line
69
[image: image.png]
[image: image.png]
Thank you,
Volodymyr
On Tue, Dec 17, 2024 at 11:17 AM Xiaoyun Zhang @.***>
wrote:

@dieron https://github.com/dieron and @faikozcan
https://github.com/faikozcan We push a fix for support arm64. Here's
the vsix artifact, would you please help us a favor and see if it works on
your arm device?

https://drive.google.com/file/d/1nsQnyrt6pH0zjxR1g_ohkM55zzD1A1pT/view?usp=sharing
—
Reply to this email directly, view it on GitHub
https://github.com/dotnet/machinelearning-modelbuilder/issues/2982#issuecomment-2549400748,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAR7CIBK3MOS2QCDNSIPFT2GB2DVAVCNFSM6AAAAABRPSFVBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBZGQYDANZUHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

@dieron Thanks for confirming that. Would you pls help us a favor and do one more check while I find an arm64 device to debug?

Uninstall Model Builder component from VS installer

Reinstall the vsix I share above.

This is to make sure that the new automl service from updated vsix get registered in Visual studio

No, it didn't help. I uninstalled the extension and the component in VS
Installer. But it's still the same error.
Volodymyr
On Wed, Dec 18, 2024 at 3:21 PM Xiaoyun Zhang @.***>
wrote:

@dieron https://github.com/dieron Thanks for confirming that. Would you
pls help us a favor and do one more check while I find an arm64 device to
debug?

Uninstall Model Builder component from VS installer

image.png (view on web)
https://github.com/user-attachments/assets/14b9818a-1363-47a8-a29b-94988952ae4b

Reinstall the vsix I share above.

This is to make sure that the new automl service from updated vsix get
registered in Visual studio
—
Reply to this email directly, view it on GitHub
https://github.com/dotnet/machinelearning-modelbuilder/issues/2982#issuecomment-2552451147,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAR7COALLQ6U4A57TSTSK32GH7PZAVCNFSM6AAAAABRPSFVBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJSGQ2TCMJUG4
.
You are receiving this because you were mentioned.Message ID:
@.***>


typo

used in a contant expression -> used in a constant expression

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c017cf85-7a65-0bbb-b5e1-c6e5ca93ee3b
Version Independent ID: 78f89035-bc4b-61bc-629d-50307b539b83
Content: constexpr lambda expressions in C++
Content Source: docs/cpp/lambda-expressions-constexpr.md
Product: visual-cpp
Technology: cpp-language
GitHub Login: @mikeblome
Microsoft Alias: mblome

@otyshev Fixed. Thanks for pointing it out.


fix: Undefined variable errors in Vim script

Utility Vim script function for applyTextEdit may give "E121: Undefined variable" errors.  This PR fixes them.

(Could you please publish a new release on jsr.io again once this PR is merged?)

thanks!
published v0.10.1.

Thank you


Create, submit and maintain a sitemap.xml for AMP By Example

We already autogenerate the sitemap: https://ampbyexample.com/sitemap.xml
I don't know if we are submitting it to Google/others. @sebastianbenz ?

It's picked up automatically by google. Closing as already implemented.


Reduced remote_timeout to 15s

Reduced the remote_timeout from 30s to 15s, if the remote endpoint is unreachable Prometheus should now reload faster
Issue #12761

Hi there, I appreciate the PR trying to fix this issue. However, what you're doing here is just modifying a file used for testing configuration validation, which doesn't have any effect on the program itself. Moreover, this value, remote_timeout can be configured anyway, so there's little need to modify it in the code unless its default value is causing problems.

Hi @LeviHarrison, sorry for the PR not meeting expectations, I asked the person who opened this issue about where should I be headed to look into this issue and he pointed me to look into queue_manager.go then I went to there and was looking for the part that might help me with the issue where I discovered a function func sendWriteRequestWithBackoff I tried my best to find out if something needs to be changed there but I was unable to do it meanwhile I also went through documentation in order to know more about this, it is there only where I found remote_timeout and I thought maybe this can help so I opened PR as I wanted to first see if the altering configurations work or not, hardcoding was the last solution I thought off.
I've just begun contributing to open source and I don't know that much but I am looking forward to learn and contribute and I really want to enhance my skills by fixing this issue, can you please highlight the key areas and that should be taken into consideration for a beginner like me to contribute and fix this issue, I am willing to learn whatever it takes to contribute.

Hi @LeviHarrison, I was searching through the code to figure out where do I need to make changes, it is when I came across
web.enable-lifecycle which is disabled by default and controls HTTP reloads and shutdowns of Prometheus as mentioned in the  official documentation, so if I set it to true is it going to fix this issue ?

If you want to fix #12761, you should try to recreate the symptoms described there.
That is, when you start Prometheus, there is a 2-minute wait.
Then if you've fixed it, when you start Prometheus there should be a much shorter wait.
You can try that out now on this branch.  (Since you didn't change the code of Prometheus, I expect no change in behaviour)

I am trying that but really having hard time building prometheus onto my system to test my changes, I am very new to Open Source and don't really know that much but I am willing to learn and get things done.

I really appreciate the effort, but it is essential to be able to build and run the program.
What kinds of troubles are you having?
[Personally I struggle with the build running npm; only one of my machines has that installed and I don't know anything about front-end development. I wish there was a way to skip that, maybe at the cost of a reduced UI experience.]

Seems like we are on the same page, I am also facing the issue with npm

Hi @LeviHarrison, I built prometheus onto my local machine and executed the prometheus.yml with the configurations as mentioned in the issue, but the logs that I am seeing doesn't match with logs provided in the issue, can you please help me out
Logs as mentioned in issue:
ts=2023-08-25T11:56:44.636Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="Stopping remote storage..."
ts=2023-08-25T11:57:27.015Z caller=dedupe.go:112 component=remote level=warn remote_name=test url=http://<IP_ADDRESS> msg="Failed to send batch, retrying" err="Post \"http://<IP_ADDRESS>\": dial tcp <IP_ADDRESS>:80: connect: no route to host"
ts=2023-08-25T11:57:44.637Z caller=dedupe.go:112 component=remote level=error remote_name=test url=http://<IP_ADDRESS> msg="non-recoverable error" count=2000 exemplarCount=0 err="context canceled"
ts=2023-08-25T11:57:44.658Z caller=dedupe.go:112 component=remote level=error remote_name=test url=http://<IP_ADDRESS> msg="Failed to flush all samples on shutdown" count=7999
ts=2023-08-25T11:57:44.659Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="WAL watcher stopped" queue=test
ts=2023-08-25T11:57:44.659Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="Stopping metadata watcher..."
ts=2023-08-25T11:58:34.482Z caller=dedupe.go:112 component=remote level=warn remote_name=test url=http://<IP_ADDRESS> msg="Failed to send batch, retrying" err="Post \"http://<IP_ADDRESS>\": dial tcp <IP_ADDRESS>:80: connect: no route to host"
ts=2023-08-25T11:58:44.659Z caller=dedupe.go:112 component=remote level=error remote_name=test url=http://<IP_ADDRESS> msg="Failed to flush metadata"
ts=2023-08-25T11:58:44.660Z caller=dedupe.go:112 component=remote level=warn remote_name=test url=http://<IP_ADDRESS> msg="Failed to send batch, retrying" err="Post \"http://<IP_ADDRESS>\": context canceled"
ts=2023-08-25T11:58:44.660Z caller=dedupe.go:112 component=remote level=error remote_name=test url=http://<IP_ADDRESS> msg="non-recoverable error while sending metadata" count=701 err="context canceled"
ts=2023-08-25T11:58:44.660Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="Scraped metadata watcher stopped"
ts=2023-08-25T11:58:44.712Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="Remote storage stopped."
ts=2023-08-25T11:58:44.714Z caller=manager.go:677 level=warn component="rule manager" file=/etc/prometheus/conf/rules/smf.yaml group=smf name=smf-n4c-down index=0 msg="Evaluating rule failed" rule="alert: smf-n4c-down\nexpr: pfcp_node_status{target_type=\"smf\"} < 1\nlabels:\n  severity: warning\nannotations:\n  description: SMF unable to get service from UPF {{ $labels.remote_ip }}, due to\n    connectivity failure or insufficient UPF resources.\n  summary: SMF unable to get service from UPF {{ $labels.remote_ip }}\n" err="query timed out in expression evaluation"
ts=2023-08-25T11:58:44.761Z caller=main.go:1268 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/conf/prometheus.yaml totalDuration=2m0.126625265s db_storage=7.583µs remote_storage=2m0.076991599s web_handler=2.916µs query_engine=4.666µs scrape=401.042µs scrape_sd=282.625µs notify=23.219875ms notify_sd=104.417µs rules=24.544792ms tracing=29.75µs

Logs in my machine:
sheikhabubaker@DESKTOP-SQEJLFP:/mnt/g/Hacktoberfest_3/prometheus$ ./prometheus --config.file=prometheus.yml
ts=2023-10-16T13:18:56.246Z caller=main.go:540 level=info msg="No time or size retention was set so using the default time retention" duration=15d
ts=2023-10-16T13:18:56.247Z caller=main.go:584 level=info msg="Starting Prometheus Server" mode=server version="(version=2.47.1, branch=remote_storage, revision=8240c7a3cc62ce08827a8a53d9f15d6d3740508b)"
ts=2023-10-16T13:18:56.247Z caller=main.go:589 level=info build_context="(go=go1.20, platform=linux/amd64, user=sheikhabubaker@DESKTOP-SQEJLFP, date=20231016-12:46:46, tags=netgo,builtinassets,stringlabels)"
ts=2023-10-16T13:18:56.247Z caller=main.go:590 level=info host_details="(Linux <IP_ADDRESS>-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 DESKTOP-SQEJLFP )"
ts=2023-10-16T13:18:56.247Z caller=main.go:591 level=info fd_limits="(soft=1048576, hard=1048576)"
ts=2023-10-16T13:18:56.247Z caller=main.go:592 level=info vm_limits="(soft=unlimited, hard=unlimited)"
ts=2023-10-16T13:18:56.261Z caller=web.go:566 level=info component=web msg="Start listening for connections" address=<IP_ADDRESS>:9090
ts=2023-10-16T13:18:56.264Z caller=main.go:1025 level=info msg="Starting TSDB ..."
ts=2023-10-16T13:18:56.270Z caller=tls_config.go:274 level=info component=web msg="Listening on" address=[::]:9090
ts=2023-10-16T13:18:56.272Z caller=tls_config.go:277 level=info component=web msg="TLS is disabled." http2=false address=[::]:9090
ts=2023-10-16T13:18:56.292Z caller=head.go:601 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any"
ts=2023-10-16T13:18:56.292Z caller=head.go:682 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.945µs
ts=2023-10-16T13:18:56.292Z caller=head.go:690 level=info component=tsdb msg="Replaying WAL, this may take a while"
ts=2023-10-16T13:18:56.299Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=13
ts=2023-10-16T13:18:56.301Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=13
ts=2023-10-16T13:18:56.302Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=13
ts=2023-10-16T13:18:56.304Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=13
ts=2023-10-16T13:18:56.306Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=13
ts=2023-10-16T13:18:56.308Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=5 maxSegment=13
ts=2023-10-16T13:18:56.310Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=6 maxSegment=13
ts=2023-10-16T13:18:56.313Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=7 maxSegment=13
ts=2023-10-16T13:18:56.317Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=8 maxSegment=13
ts=2023-10-16T13:18:56.319Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=9 maxSegment=13
ts=2023-10-16T13:18:56.320Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=10 maxSegment=13
ts=2023-10-16T13:18:56.322Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=11 maxSegment=13
ts=2023-10-16T13:18:56.323Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=12 maxSegment=13
ts=2023-10-16T13:18:56.324Z caller=head.go:761 level=info component=tsdb msg="WAL segment loaded" segment=13 maxSegment=13
ts=2023-10-16T13:18:56.324Z caller=head.go:798 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=2.37191ms wal_replay_duration=29.739815ms wbl_replay_duration=131ns total_replay_duration=32.139228ms
ts=2023-10-16T13:18:56.326Z caller=main.go:1046 level=info fs_type=1021997
ts=2023-10-16T13:18:56.326Z caller=main.go:1049 level=info msg="TSDB started"
ts=2023-10-16T13:18:56.326Z caller=main.go:1230 level=info msg="Loading configuration file" filename=prometheus.yml
ts=2023-10-16T13:18:56.332Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="Starting WAL watcher" queue=test
ts=2023-10-16T13:18:56.332Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="Starting scraped metadata watcher"
ts=2023-10-16T13:18:56.334Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://<IP_ADDRESS> msg="Replaying WAL" queue=test
ts=2023-10-16T13:18:56.336Z caller=main.go:1267 level=info msg="Completed loading of configuration file" filename=prometheus.yml totalDuration=9.322173ms db_storage=1.062µs remote_storage=1.676339ms web_handler=410ns query_engine=1.172µs scrape=339.38µs scrape_sd=2.761372ms notify=21.211µs notify_sd=8.587µs rules=9.468µs tracing=10.6µs
ts=2023-10-16T13:18:56.336Z caller=main.go:1010 level=info msg="Server is ready to receive web requests."
ts=2023-10-16T13:18:56.336Z caller=manager.go:1012 level=info component="rule manager" msg="Starting rule manager..."
ts=2023-10-16T13:18:56.338Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:19:56.341Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:20:56.340Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:21:56.341Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:22:56.341Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:23:56.341Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:24:56.340Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:25:56.341Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
ts=2023-10-16T13:26:56.340Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=test msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"

I can see a couple of things.  First, the original issue's logs start from a point where the config is changed and reloaded, that's why there are some "Stopping" messages.
Then, the issue talks about an "endpoint that is not reachable", whereas your "connect: connection refused" message means your endpoint is reachable and responded immediately.  That may not be important; do the config change and reload and see how close you get.

I can see a couple of things. First, the original issue's logs start from a point where the config is changed and reloaded, that's why there are some "Stopping" messages.
Then, the issue talks about an "endpoint that is not reachable", whereas your "connect: connection refused" message means your endpoint is reachable and responded immediately. That may not be important; do the config change and reload and see how close you get.

@bboreham I am still unable to recreate this issue can you please have a look and help me out with config
Prometheus version:
sheikhabubaker@DESKTOP-TFVM3PT:~/prometheus-2.46.0.linux-amd64$ ./prometheus --version
prometheus, version 2.46.0 (branch: HEAD, revision: cbb69e51423565ec40f46e74f4ff2dbb3b7fb4f0)
  build user:       root@42454fc0f41e
  build date:       20230725-12:31:24
  go version:       go1.20.6
  platform:         linux/amd64
  tags:             netgo,builtinassets,stringlabels

Config file:
  scrape_interval: 15s
  evaluation_interval: 15s
  external_labels:
remote_write:
  - name: test
    url: 'http://remote_host:9091'
    tls_config:
      insecure_skip_verify: false

rule_files:
  - /etc/prometheus/conf/rules/*.yaml

alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - '<IP_ADDRESS>:9093'

scrape_configs:
  - job_name: '****'
    scrape_interval: 10s
    http_sd_configs:
      - url: http://<IP_ADDRESS>:8089/prometheus
        refresh_interval: 60s

Logs:
sheikhabubaker@DESKTOP-TFVM3PT:~/prometheus-2.46.0.linux-amd64$ ./prometheus
ts=2024-01-06T19:31:18.139Z caller=main.go:541 level=info msg="No time or size retention was set so using the default time retention" duration=15d
ts=2024-01-06T19:31:18.139Z caller=main.go:585 level=info msg="Starting Prometheus Server" mode=server version="(version=2.46.0, branch=HEAD, revision=cbb69e51423565ec40f46e74f4ff2dbb3b7fb4f0)"
ts=2024-01-06T19:31:18.139Z caller=main.go:590 level=info build_context="(go=go1.20.6, platform=linux/amd64, user=root@42454fc0f41e, date=20230725-12:31:24, tags=netgo,builtinassets,stringlabels)"
ts=2024-01-06T19:31:18.139Z caller=main.go:591 level=info host_details="(Linux <IP_ADDRESS>-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 DESKTOP-TFVM3PT )"
ts=2024-01-06T19:31:18.139Z caller=main.go:592 level=info fd_limits="(soft=1048576, hard=1048576)"
ts=2024-01-06T19:31:18.139Z caller=main.go:593 level=info vm_limits="(soft=unlimited, hard=unlimited)"
ts=2024-01-06T19:31:18.140Z caller=web.go:563 level=info component=web msg="Start listening for connections" address=<IP_ADDRESS>:9090
ts=2024-01-06T19:31:18.141Z caller=main.go:1026 level=info msg="Starting TSDB ..."
ts=2024-01-06T19:31:18.142Z caller=tls_config.go:274 level=info component=web msg="Listening on" address=[::]:9090
ts=2024-01-06T19:31:18.142Z caller=tls_config.go:277 level=info component=web msg="TLS is disabled." http2=false address=[::]:9090
ts=2024-01-06T19:31:18.143Z caller=head.go:595 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any"
ts=2024-01-06T19:31:18.143Z caller=head.go:676 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.262µs
ts=2024-01-06T19:31:18.143Z caller=head.go:684 level=info component=tsdb msg="Replaying WAL, this may take a while"
ts=2024-01-06T19:31:18.144Z caller=head.go:755 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=2
ts=2024-01-06T19:31:18.144Z caller=head.go:755 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=2
ts=2024-01-06T19:31:18.145Z caller=head.go:755 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=2
ts=2024-01-06T19:31:18.145Z caller=head.go:792 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=20.559µs wal_replay_duration=1.976662ms wbl_replay_duration=130ns total_replay_duration=2.028572ms
ts=2024-01-06T19:31:18.145Z caller=main.go:1047 level=info fs_type=EXT4_SUPER_MAGIC
ts=2024-01-06T19:31:18.145Z caller=main.go:1050 level=info msg="TSDB started"
ts=2024-01-06T19:31:18.145Z caller=main.go:1231 level=info msg="Loading configuration file" filename=prometheus.yml
ts=2024-01-06T19:31:18.146Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="Starting WAL watcher" queue=test
ts=2024-01-06T19:31:18.146Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="Starting scraped metadata watcher"
ts=2024-01-06T19:31:18.146Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="Replaying WAL" queue=test
ts=2024-01-06T19:31:18.146Z caller=main.go:1268 level=info msg="Completed loading of configuration file" filename=prometheus.yml totalDuration=1.296618ms db_storage=1.002µs remote_storage=322.739µs web_handler=821ns query_engine=681ns scrape=684.954µs scrape_sd=67.34µs notify=10.05µs notify_sd=5.751µs rules=4.98µs tracing=5.22µs
ts=2024-01-06T19:31:18.147Z caller=main.go:1011 level=info msg="Server is ready to receive web requests."
ts=2024-01-06T19:31:18.147Z caller=manager.go:1009 level=info component="rule manager" msg="Starting rule manager..."
ts=2024-01-06T19:31:18.147Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=**** msg="Unable to refresh target groups" err="Get \"http://<IP_ADDRESS>:8089/prometheus\": dial tcp <IP_ADDRESS>:8089: connect: connection refused"
^Cts=2024-01-06T19:31:23.546Z caller=main.go:861 level=warn msg="Received SIGTERM, exiting gracefully..."
ts=2024-01-06T19:31:23.547Z caller=main.go:885 level=info msg="Stopping scrape discovery manager..."
ts=2024-01-06T19:31:23.547Z caller=main.go:899 level=info msg="Stopping notify discovery manager..."
ts=2024-01-06T19:31:23.547Z caller=manager.go:1023 level=info component="rule manager" msg="Stopping rule manager..."
ts=2024-01-06T19:31:23.547Z caller=manager.go:1033 level=info component="rule manager" msg="Rule manager stopped"
ts=2024-01-06T19:31:23.547Z caller=main.go:895 level=info msg="Notify discovery manager stopped"
ts=2024-01-06T19:31:23.547Z caller=main.go:936 level=info msg="Stopping scrape manager..."
ts=2024-01-06T19:31:23.547Z caller=main.go:881 level=info msg="Scrape discovery manager stopped"
ts=2024-01-06T19:31:23.547Z caller=main.go:928 level=info msg="Scrape manager stopped"
ts=2024-01-06T19:31:23.565Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="Stopping remote storage..."
ts=2024-01-06T19:31:23.565Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="WAL watcher stopped" queue=test
ts=2024-01-06T19:31:23.565Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="Stopping metadata watcher..."
ts=2024-01-06T19:31:23.565Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="Scraped metadata watcher stopped"
ts=2024-01-06T19:31:23.565Z caller=dedupe.go:112 component=remote level=info remote_name=test url=http://remote_host:9091 msg="Remote storage stopped."
ts=2024-01-06T19:31:23.565Z caller=notifier.go:603 level=info component=notifier msg="Stopping notification manager..."
ts=2024-01-06T19:31:23.565Z caller=main.go:1157 level=info msg="Notifier manager stopped"
ts=2024-01-06T19:31:23.565Z caller=main.go:1169 level=info msg="See you next time!"

Did you do the reload?  From the original issue:

Configure a "remote_write" endpoint that is not reachable
Remove that endpoint from the config
Reload using API /reload

You need to add --web.enable-lifecycle to the command-line, and run something like:
curl -X POST :9090/-/reload

getting very difficult to setup the mentioned Linux version.

The exact version is unlikely to be important.  Doing the three steps listed is very important.

Did you do the reload? From the original issue:

Configure a "remote_write" endpoint that is not reachable
Remove that endpoint from the config
Reload using API /reload

You need to add --web.enable-lifecycle to the command-line, and run something like:
curl -X POST :9090/-/reload

did you mean like this ./prometheus --web.enable-lifecycle ?

Ok and did you do all three steps?

Ok and did you do all three steps?

Yes I did it

Good, making some progress.
As you may notice it isn’t reproducing the issue reported.
This might be because your Prometheus isn’t scraping any data, because you have pointed it at a nonexistent http_sd_configs.
You have data in the WAL, so it was scraping in the past. Can you get it back to that state to try to get a bit closer to the original issue?

I made a comment on the initial issue: #12761
That being said, I'm not seeing changes here other than config flag?

I made a comment on the initial issue: #12761
That being said, I'm not seeing changes here other than config flag?

This PR is still in development and I'm debugging into query_manager.go to find the solution, as soon as I get a solution I will push  the latest commits and restore the config flag changes to it's previous state.

That being said, I'm not seeing changes here other than config flag?

That’s right, see early comments.  @Sheikh-Abubaker is coming up to speed on the issue.
@Sheikh-Abubaker the next step would be to figure out where it’s stuck in the 2-minute window. Can you run Prometheus under a debugger?

@Sheikh-Abubaker the next step would be to figure out where it’s stuck in the 2-minute window. Can you run Prometheus under a debugger?

@bboreham yes I'm trying, but the thing when I open the prometheus directory as workspace folder the debugger throws error that it can't find any go files in the main prometheus directory, and when opening prometheus/storage/remote it throws error that the file is not executable, then I tried copying all the contents of query_manager.go into a new separate directory adding prometheus.yml in the same new directory and run debugger after resolving some the warnings I was able to get the debugger up and running but then after some time when I repeated the same steps creating new directory with query_manager,go and prometheus.yml the debugger won't start anymore I'm trying to fix errors and get debugger up and running again.
Sorry for taking so long, I've not used a debugger before so I'm learning and implementing that, also as an engineering student I've to manage both university studies and Open Source, please give me some time, I'll surely find something with debugging this, I'm trying everyday to find out the solution.

@Sheikh-Abubaker the next step would be to figure out where it’s stuck in the 2-minute window. Can you run Prometheus under a debugger?

@bboreham yes I'm trying, but the thing when I open the prometheus directory as workspace folder the debugger throws error that it can't find any go files in the main prometheus directory, and when opening prometheus/storage/remote it throws error that the file is not executable, then I tried copying all the contents of query_manager.go into a new separate directory adding prometheus.yml in the same new directory and run debugger after resolving some the warnings I was able to get the debugger up and running but and successfully debugged some of the parts but then after some time when I repeated the same steps creating new directory with query_manager,go and prometheus.yml the debugger won't start anymore I'm trying to fix errors and get debugger up and running again.
Sorry for taking so long, I've not used a debugger before so I'm learning and implementing that, also as an engineering student I've to manage both university studies and Open Source, please give me some time, I'll surely find something with debugging this, I'm trying everyday to find out the solution.

@Sheikh-Abubaker the next step would be to figure out where it’s stuck in the 2-minute window. Can you run Prometheus under a debugger?

@bboreham yes I'm trying, but the thing when I open the prometheus directory as workspace folder the debugger throws error that it can't find any go files in the main prometheus directory, and when opening prometheus/storage/remote it throws error that the file is not executable, then I tried copying all the contents of query_manager.go into a new separate directory adding prometheus.yml in the same new directory and run debugger after resolving some the warnings I was able to get the debugger up and running and successfully debugged some of the contents but then after some time when I repeated the same steps creating new directory with query_manager.go and prometheus.yml the debugger won't start anymore I'm trying to fix errors and get debugger up and running again.
Sorry for taking so long, I've not used a debugger before so I'm learning and implementing that, also as an engineering student I've to manage both university studies and Open Source, please give me some time, I'll surely find something with debugging this, I'm trying everyday to find out the solution.

@Sheikh-Abubaker rest assured I understand you are learning, and it's totally fine to take time.

This PR is still in development and I'm debugging into query_manager.go to find the solution, as soon as I get a solution I will push the latest commits and restore the config flag changes to it's previous state.

No worries at all. And thanks to @bboreham for guiding you along here. Feel free to ping me directly here or on the CNCF slack (or Grafana Labs public slack) if you have questions about bits of code in storage/remote.
As for the 2m duration, it sounds like we retry somewhere at least one time since the flush deadline we use on shutdown has a default value of 1m here and the shutdown logic is here.
Regarding the debugger, I've found that go debuggers on the CLI or in VSCode can be hard to setup and get working properly but for me at least the debugger worked out of the box easily with Goland, which students can get a free license for here.

This PR is still in development and I'm debugging into query_manager.go to find the solution, as soon as I get a solution I will push the latest commits and restore the config flag changes to it's previous state.

No worries at all. And thanks to @bboreham for guiding you along here. Feel free to ping me directly here or on the CNCF slack (or Grafana Labs public slack) if you have questions about bits of code in storage/remote.
As for the 2m duration, it sounds like we retry somewhere at least one time since the flush deadline we use on shutdown has a default value of 1m here and the shutdown logic is here.
Regarding the debugger, I've found that go debuggers on the CLI or in VSCode can be hard to setup and get working properly but for me at least the debugger worked out of the box easily with Goland, which students can get a free license for here.

@cstyan
Thank you so much for the valuable information above, and yes go debugger in VS code is really giving me a hard time, I'll surely checkout the above links.

@bboreham I've successfully configured the debugger to run prometheus, but now I am wondering how do I perform a config reload now ?

@bboreham I wanted to confirm one thing, this prometheus repo that I have cloned doesn't have a executable binary right ? to get the binary I have to build prometheus into my local system then run that binary within a debugger and look into call stack ?

@Sheikh-Abubaker that's right, the built binary isn't checked into the repo. Run make build.
As for config reloads it's documented in the first section here, specfically
Prometheus can reload its configuration at runtime.
If the new configuration is not well-formed, the changes will not be applied.
A configuration reload is triggered by sending a SIGHUP to the Prometheus process or sending a HTTP POST request to the /-/reload endpoint (when the --web.enable-lifecycle flag is enabled).
This will also reload any configured rule files.

@cstyan Thanks for the valuable information.

@bboreham I've successfully built the prometheus binary, but when running prometheus under the debugger I'm unable to recreate the issue inside debugger, I've been researching about how to perform a config reload under debugger but didn't yet found a way to do it, if you know a way to do it can you please tell me.

@bboreham I've successfully built the prometheus binary, but when running prometheus under the debugger I'm unable to recreate the issue inside debugger, I've been researching about how to perform a config reload under debugger but didn't yet found a way to do it, if you know a way to do it can you please tell me.

Sometimes fighting with the debugger may be more trouble than it's worth, especially if you have limited time to try and work on things here. I'd suggest resorting to our old friend fmt.Println for now, also if you ctrl + \ on the CLI it will cause the process to exit and dump the current go routines call stacks. That way you can look for one that's waiting on a channel, timer, etc.

Sometimes fighting with the debugger may be more trouble than it's worth, especially if you have limited time to try and work on things here. I'd suggest resorting to our old friend fmt.Println for now, also if you ctrl + \ on the CLI it will cause the process to exit and dump the current go routines call stacks. That way you can look for one that's waiting on a channel, timer, etc.

Thank you.

The remote write shards will attempt to flush all of their data before exiting, as a way to avoid losing data, which is what would be happening here if you have prometheus running with remote write configured (especially if the endpoint you configured it to write to isn't actually up and running).
The reshard loop should exit as soon as an exit signal is sent, either via a terminal signal or a config reload, etc. What would actually be blocking is the shutdown of the shards here: https://github.com/prometheus/prometheus/blob/cd3e0078f063e99c5ec723fdc487675dec775060/storage/remote/queue_manager.go#L1200-L1233

So could it be done this way that we introduce a conditional statement within reshard loop like if it recieves an exit signal either via a terminal signal or a config reload it should break the loop else it should go for softshutdown ?

I'm not sure I understand your question, the reshard loop will already exit as soon as it receives the exit signal over the channel. That happens here.

@cstyan
https://github.com/prometheus/prometheus/blob/cd3e0078f063e99c5ec723fdc487675dec775060/storage/remote/queue_manager.go#L1215-L1217
This above for loop sends signal after flushing the samples via s.done to the below select case of s.done right ? and then it proceeds to the hardshutdown part  after one of the conditions for the select case is satisfied right?
https://github.com/prometheus/prometheus/blob/cd3e0078f063e99c5ec723fdc487675dec775060/storage/remote/queue_manager.go#L1219

Unless I am misremembering, the original goal here was to try and reduce the time it takes to reload prometheus' config if the remote endpoint is down and samples are not successfully being sent. The config reload is done by write.go ApplyConfig, where here we call queue.Stop() https://github.com/prometheus/prometheus/blob/cd3e0078f063e99c5ec723fdc487675dec775060/storage/remote/write.go#L215-L219
queue.Stop() closes the quit channel, which will cause reshardLoop to exit immediately on it's next iteration https://github.com/prometheus/prometheus/blob/cd3e0078f063e99c5ec723fdc487675dec775060/storage/remote/queue_manager.go#L1129
This :point_up: is what you were previously asking about here, that's not where we'd be blocking. Flushing the queues is what we could be blocking on, the line you linked to here https://github.com/prometheus/prometheus/blob/cd3e0078f063e99c5ec723fdc487675dec775060/storage/remote/queue_manager.go#L1216
However, that behaviour is intentional. Just because the endpoint is currently down at the time a shutdown is initiated doesn't mean it won't come back immediately after that, allowing for proper flushing of the enqueued samples. IMO The real fix is the following two pieces of work:

https://github.com/prometheus/prometheus/pull/13769
https://github.com/prometheus/prometheus/issues/8809

Unless I am misremembering, the original goal here was to try and reduce the time it takes to reload prometheus' config if the remote endpoint is down and samples are not successfully being sent.

Yes I was just verifying if my understanding is correct there.

@cstyan Since I haven't replied for a while just updating I'm still interested and really want to contribute to this one, being a student, I'm trying to give as much time as possible to research and understand the codebase.

@Sheikh-Abubaker no problem, there's no rush. I think maybe it makes sense to close this PR and for you to participate in the conversations on these:

https://github.com/prometheus/prometheus/issues/7230
https://github.com/prometheus/prometheus/pull/13769
https://github.com/prometheus/prometheus/issues/8809

What do you think?

What do you think?
Yeah sure 👍!!


[Bug]: Pipeline imported failed from towhee on OS ubuntu

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Pipeline imported failed from towhee on OS ubuntu
>>> from towhee import pipeline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/__init__.py", line 22, in <module>
    from towhee.engine.engine import Engine, start_engine
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/engine.py", line 18, in <module>
    from towhee.engine.pipeline import Pipeline
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/pipeline.py", line 17, in <module>
    from towhee.engine.graph_context import GraphContext
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/graph_context.py", line 21, in <module>
    from towhee.engine.operator_context import OperatorContext, OpStatus
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/operator_context.py", line 19, in <module>
    from towhee.engine.operator_runner.runner_base import RunnerStatus
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/operator_runner/__init__.py", line 22, in <module>
    from towhee.engine.operator_io.reader import DataFrameReader
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/operator_io/__init__.py", line 16, in <module>
    import towhee.engine.operator_io.reader as io_reader
AttributeError: module 'towhee.engine' has no attribute 'operator_io'

Expected Behavior
pipeline imported from towhee  successfully
Steps To Reproduce
from towhee import pipeline

Environment
- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2.3.dev3
- OS(Ubuntu or CentOS): Ubuntu 18.04
- CPU/Memory:
- GPU:
- Others:

Anything else?
No response

I believe that this is with respect to the main branch, because the pip version works fine. Would you mind sharing how I can install the main branch locally?

@Ashvith
It is an issue when using python3.6, and now it is fixed by pr #311.
And verified, it is fixed now. Thanks.


Remove listener if one already exists

This commit makes it so EventListener::listen() does not overwrite
existing listeners if they already exist. If the listener is already
registered in an event listener list, it removes the listener from that
list before registering the new listener. This soundness bug was missed
during #94.
This is a patch-compatible fix for #100. We may also want an API-level
fix for #100 as well. This is up for further discussion.

Slight benchmark regression, but it's fine for now


Http Client suggestion may be changed

On your README page we can read

But as the Socket Client is now deprecated, I suggest to replace it by the cURL client

The new recommendation should give something like :

A general suggestion for HTTP Client, using php-http/curl-client and nyholm/psr7:
composer require php-http/curl-client nyholm/psr7

Neither packagist nor the repository shows anything about it being deprecated.
It actually had a release 2 months ago. Am I missing anything?

The deprecations are visible on documentation at https://docs.php-http.org/en/latest/clients.html

@flavioheleno Coul you explain your choice of https://packagist.org/packages/kriswallsmith/buzz rather than https://packagist.org/packages/php-http/curl-client (as I've suggested) ?

hey @llaville, not in any particular order:

buzz has both a curl-based client and a stream-based client, which makes ext-curl fully optional;
buzz has been around longer than curl-client;
buzz has a backwards compatibility promise;
the fact that socket-client's deprecation notice was only displayed on its website and not on packagist nor its repository is, in my opinion, a poor practice towards users.


Suggested way to handle read-only concurrency

In the use case that I have a query that I saved as a materialized view -- or let's just say a table called SavedQuery. Let's say I want to grab some statistics for each column, and there are ten columns. So the queries that I run are:
SELECT <info> FROM SavedQuery GROUP BY <field1>
SELECT <info> FROM SavedQuery GROUP BY <field2>
...

In the case that I run all ten queries in parallel (or whatever batch size I decide to use), what is the suggested way to run these queries concurrently?

Is this a question for DuckDB or DuckDB wasm? If the latter, you can submit queries with independent connections but keep in mind that the memory usage will increase as the query results have to be kept in memory.

Also note that your example is the perfect example for grouping sets and can be done in a single query.
SELECT <info> FROM SavedQuery GROUP BY GROUPING SETS ((<field1>), (<field2>))

Right now, DuckDB runs single-threaded in the web worker which means that concurrency right now can only really reduce the overhead of the message passing to and from the worker.
E.g. if you would do:
inflight.push(conn1.query('foo'));
inflight.push(conn2.query('bar'));
await Promise.all(inflight);

it would still execute foo and bar in sequence at the moment just because we're running with 1 thread without SharedArrayBuffers. That would also profit from a fully async DuckDB plan driver but we don't have that one yet.
It would, however, get rid of these additional event loop ticks that await for the response of foo before sending the next query.
Right now, it would even work to fire the queries over a single connection:
inflight.push(conn1.query('foo'));
inflight.push(conn1.query('bar'));
await Promise.all(inflight);

But this may break in the future. For example, using send instead of query here would be malformed since the connection is used to hold the query results (and stream them back to the main thread in a streaming fashion.

In background process, a query is running like CREATE TABLE IF NOT EXISTS tablename AS (SELECT * FROM "sample.csv");
the sample.csv has 10m csv file.
When run another query with independent connection like select 1, it takes 10s
i think the connections is not independent. What is the solution?

Yes, DuckDB-Wasm can only execute one query at a time right now. We're looking into single-threaded concurrency in the web worker but this is not possible today.

@ankoh I see. Yea, GROUPING SETS is great in that particular use case. I suppose another one might be a dashboard where there are five different graphs/panels/etc and the queries would be better rendered in parallel (or 'batches' might be a better word if we want to limit the concurrency level).
As far as executing some single-threaded concurrency, is that something that occurs mostly on the DuckDB-wasm side or would need modifications in DuckDB itself? Is it a particularly complex item to implement? I wonder what the throughput of copying the Arrow buffer from one worker to another would be (If I recall correctly, I remember it taking a not-insignificant time to copy over the data when last testing something similar).

@domoritz

Is this a question for DuckDB or DuckDB wasm? If the latter, you can submit queries with independent connections but keep in mind that the memory usage will increase as the query results have to be kept in memory.

I meant specifically for the DuckDB-Wasm use case.

@domoritz

Is this a question for DuckDB or DuckDB wasm? If the latter, you can submit queries with independent connections but keep in mind that the memory usage will increase as the query results have to be kept in memory.

I meant specifically for the DuckDB-Wasm use case. Just to clarify, do all connections occur within the same web-worker as long as they're tied to the same duckdb.AsyncDuckDB instance?

Yes they do

@domoritz

Is this a question for DuckDB or DuckDB wasm? If the latter, you can submit queries with independent connections but keep in mind that the memory usage will increase as the query results have to be kept in memory.

I meant specifically for the DuckDB-Wasm use case. Just to clarify, do all connections occur within the same web-worker as long as they're tied to the same duckdb.AsyncDuckDB instance?

It's mostly on the DuckDB side. In DuckDB, you currently start a query and wait for it blockingly to finish. Consequently, if you have only the single thread of the Web worker, you're limited to executing 1 query at a time in your instance, independent of the connections. We're currently working on lifting this since this is tightly coupled with the asynchronous I/O story but that's the state of today.

I wonder what the throughput of copying the Arrow buffer from one worker to another would be (If I recall correctly, I remember it taking a not-insignificant time to copy over the data when last testing something similar).

The cost for copying is not zero but also not too severe.
The packing of the arrow buffer and reading everything again will hurt more than passing it to the worker.
Especially since you can pass the underlying ArrayBuffer as Transferable to postMessage.

@ankoh I see. Yea, GROUPING SETS is great in that particular use case. I suppose another one might be a dashboard where there are five different graphs/panels/etc and the queries would be better rendered in parallel (or 'batches' might be a better word if we want to limit the concurrency level).

Yes, this will come up when rendering 5 graphs in parallel. My advice here would be for now to just run the queries on different connections and await them independently. They will be executed one at a time but you don't need to serialize the query calls. Eventually, we'll be able to execute multiple queries concurrently on a single thread which will just speedup your execution without further adjustments on your end.

Closing this for now. Feel free to open a follow up discussion.


XD-3376 Gemfire Sink

Implements a Gemfire Sink.

Changed connection properties semantics from XD:  -DhostAddresses=host1:port1,host2:port2 -DconnectType=[locator | server]

linked with the latest SDG 1.6.2 and Gemfire 8.0.0. Eventually these deps should be added at runtime, same as JDBC modules. We can downgrade the default to 7 perhaps, but upgrade currently requires rebuild per the README.

Much of this is a port of test fixture code from SDG to launch an external Gemfire Server for IntegrationTests which will be shared by all gemfire modules. I don't think it's worth reviewing here as it's used extensively in the SDG build.

Much of this is a port of test fixture code from SDG to launch an external Gemfire Server for IntegrationTests which will be shared by all gemfire modules. I don't think it's worth reviewing here as it's used extensively in the SDG build.

In that case, is there hope for us to maybe re-use an artifact? Maybe SDG could publish their src/test/java classes as an artifact?

Maybe SDG could publish their src/test/java classes as an artifact?

I doubt that this will happen in the next few weeks
I will refer this to @jxblum (I don't think he will see this mention automatically). We have considered this before since spring-integration-gemfire has some similar requirements.
Note that this code for the most part has little or no dependencies on gem or SDG. It's mostly using ProcessBuilder, etc. to launch an external (JVM) process, wait for it to start and then stopping it. So as a reusable artifact it may rightfully belong elsewhere. However this is a common Gemfire testing story.

Hi Fellas-
Right, the src/test/java/org.springframework.data.gemfire.process API is a generalized (GemFire agnostic) way to launch any JVM-based process bootstrapped with a given Java class.  The API stems from ideas/work I did on GemFire (specifically, Gfsh), but is even more extensible and reusable.
The (ProcessExecutor) API requires a Java class to bootstrap the JVM process.  Currently, I have a few such classes in src/test/java/org.springframework.data.gemfire.fork such as ServerProcess, which is used to launch any Spring-configured application (again, GemFire agnostic), given a Spring ApplicationContext configuration file (XML).  An example can be seen here.
All of this is meant to replace SDG's ForkUtil class as a more capable and general utility for forking JVM processes.
There has been quite a bit of interests (even from customers, e.g. BMO) for having such classes outside of SDG to help them with their (integration) testing efforts.  However, this has been considerably lower priority than other deliverables.
I should have considerably more time after SpringOne to consider this, and I have quite a few more ideas on how to expand and make this API even more robust.
Cheers,
John

I'm testing this now with the connected car and came across an issue.  In the connected car we ran the following streams:
// Create the initial stream to take HTTP data and enrich it and store it to a local file
stream create --name IoT-HTTP --definition "http | filter --script=file:/opt/pivotal/IoT-ConnectedCar/IoT-DataFilter/src/main/resources/DataFilter.groovy | acmeenrich | shell --inputType=application/json --command='/opt/pivotal/anaconda/bin/python ./StreamPredict.py' --workingDir=/opt/pivotal/IoT-ConnectedCar/IoT-DataScience/PythonModel/ | hdfs" --deploy

// Create a tap that reads the transformed json data into gemfire
stream create --name IoT-Gemfire --definition "tap:stream:IoT-HTTP.shell > typeconversiontransformer | gemfire-server --regionName=car-position --keyExpression=payload.vin" --deploy

The typeconversiontransformer converts JSON to a java object (CarPosition in this case) and returns it.  The gemfire-server sink would then save it to Gemfire.  However, before, we didn't need to add the jar that contains CarPosition class to the sink.  In this new version it seems we do.  I'm leaning towards this being a bug with the old version (classpath leak somewhere) but not sure.

@mminella - AFAICT with XD you would still need to have the domain object in the sink class path and probably in the Gemfire (the actual server) class path. The recommended approach would be to add that jar to xd/lib. Here that is not an option.

Did not manage to have this work.
This is what I tried, using the default GemFire server we ship with XD:
stream create foo --definition "time | gemfire  --host-addresses=localhost:40404 --region-name=Stocks --key-expression=payload" --deploy

I'm getting an "EvaluationContext must not be null" exception. Also, seems like keyExpression used to be <streamname> in XD, wonder if we can/want replicate this.

what's the stack trace?
On Thu, Oct 8, 2015 at 4:22 AM, Eric Bottard<EMAIL_ADDRESS>wrote:

Did not manage to have this work.
This is what I tried, using the default GemFire server we ship with XD:
stream create foo --definition "time | gemfire  --host-addresses=localhost:40404 --region-name=Stocks --key-expression=payload" --deploy
I'm getting an "EvaluationContext must not be null" exception. Also, seems
like keyExpression used to be  in XD, wonder if we can/want
replicate this.
—
Reply to this email directly or view it on GitHub
https://github.com/spring-cloud/spring-cloud-stream-modules/pull/22#issuecomment-146453744
.

This is a snippet. Not sure if it's actually the root cause, or an exception happening when the error is going to a DLQ or similar:
java.lang.IllegalArgumentException: EvaluationContext must not be null
	at org.springframework.util.Assert.notNull(Assert.java:115) ~[spring-core-4.2.1.RELEASE.jar!/:4.2.1.RELEASE]
	at org.springframework.expression.spel.ExpressionState.<init>(ExpressionState.java:85) ~[spring-expression-4.2.1.RELEASE.jar!/:4.2.1.RELEASE]
	at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:329) ~[spring-expression-4.2.1.RELEASE.jar!/:4.2.1.RELEASE]
	at org.springframework.integration.redis.outbound.RedisQueueOutboundChannelAdapter.handleMessageInternal(RedisQueueOutboundChannelAdapter.java:115) ~[spring-integration-redis-4.2.0.RELEASE.jar!/:na]
	at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127) ~[spring-integration-core-4.2.0.RELEASE.jar!/:na]
	... 17 common frames omitted
Wrapped by: org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.integration.redis.outbound.RedisQueueOutboundChannelAdapter@57b9c7a3]; nested exception is java.lang.IllegalArgumentException: EvaluationContext must not be null
	at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:139) ~[spring-integration-core-4.2.0.RELEASE.jar!/:na]
	at org.springframework.cloud.stream.binder.redis.RedisMessageChannelBinder$1$2.recover(RedisMessageChannelBinder.java:277) ~[spring-cloud-stream-binder-redis-1.0.0.BUILD-SNAPSHOT.jar!/:1.0.0.BUILD-SNAPSHOT]
	at org.springframework.cloud.stream.binder.redis.RedisMessageChannelBinder$1$2.recover(RedisMessageChannelBinder.java:267) ~[spring-cloud-stream-binder-redis-1.0.0.BUILD-SNAPSHOT.jar!/:1.0.0.BUILD-SNAPSHOT]
	at org.springframework.retry.support.RetryTemplate.handleRetryExhausted(RetryTemplate.java:458) ~[spring-retry-1.1.2.RELEASE.jar!/:na]
	at org.sp

Hi there!
Have had a discussion with @dturanski regarding the EvaluationContext must not be null error.
Please, find this code:
https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream-binders/spring-cloud-stream-binder-redis/src/main/java/org/springframework/cloud/stream/binder/redis/RedisMessageChannelBinder.java#L167
and after that this:
https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream-binders/spring-cloud-stream-binder-spi/src/main/java/org/springframework/cloud/stream/binder/MessageChannelBinderSupport.java#L379
So, onInit() is called before the setup of evaluationContext.

@ericbottard I was able to reproduce this when there is no cache server running. BTW, you need to set --connect-type=server. Default is locator now which is a breaking change from XD, but more common in the wild.
  stream create foo --definition "time | gemfire  --host-addresses=localhost:40404 --connect-type=server --region-name=Stocks --key-expression=payload" --deploy

You need a cache server running. For this test, the one the XD distribution one is fine:
      $cd gemfire
      $bin/gemfire-server config/cq-demo.xml

The entries will be logged in the console. When you kill the server you (or don't start it, you see evaluation context cannot be null which @artembilan tracked to a bug in the way RedisMessageBinder is initialized.

@dturanski , as we talked with you yesterday: https://jira.spring.io/browse/INT-3850.
I think it will be available in the upcoming SI-4.2.1 release.
BTW, I don't see reaction to my comment regarding the bug.
Do, you need some ticket on the matter? Where ?
Or do you accept only contribution in face of PRs?
:smile:

Merged as b12eaba454a202a91668ee4dd4f7518bebed3616 (as well as https://github.com/spring-cloud/spring-cloud-dataflow/commit/392bc95f4791ce4234e1228f98ce98f043fbbd6d on s-c-d)


Sub numbers on Overlay are not updating

The number of subs on a voyage, et. al., are not updating when you add a character to the Ignored List or delete them from the configuration.

Same as #18

Closed as related to #18


Use Arrow type for both input and output of dora-ros2-bridge

This PR replaces serde-yaml with arrow. This allows to have more flexibility with types as well as having a more coherent typing system for both dora and dora-ros2-bridge.
To do this, I have to:

Replace pythonize into arrow.
Replace ser/de method to use arrow types instead of serde-yaml types.
Replace defaults mecanism as arrow typing does not have default value.

This should make further integration from dora-rs to dora-ros2-bridge easier as well as make supporting very specific type better.

I updated https://github.com/dora-rs/dora-ros2-bridge/pull/8 to include this PR.


Executing custom User autocmd commands can cause folds to close

Create the following files somewhere on your filesystem.
 The fold will be closed. If you edit setup.vim, remove the autocmd User Anything let x=3 line, and try the same again, the fold will remain open. It seems like executing custom autocmd commands can cause folds to be closed.

You need the  flag, if you don't want the modelines to be re-evalutated.

Which flag is that?

Oh, it appeared in the email from GitHub, but not the comment, due to Markdown wizardry.

Hm, <nomodeline> Github markdown swallowed it.

Okay, I'll add that flag. Thanks for the info! I didn't know it was connected to folding.


[INTERNAL] Fix verbose logging when test dir is not available

Coverage remained the same at 90.941% when pulling 8441daa56d8ce7bbc363b12e4bac35d9ab09181a on fix-logging into 38232e3e374874efe3aa9876ea4f5b5e6d57a61a on master.


MXNET 0.10.0 "endsWith" not found

I'm using MXNET 0.10.0 on Linux Mint (with GPU) and get the following error when running a CNN model using R:
Error in mx.model.init.params(symbol, input.shape, output.shape, initializer,  :
could not find function "endsWith"
I've tried downloading and recompiling the MXNET code over the past couple of days with the same results.

I guess you are using R 3.2. endsWith was introduced in R 3.3.
I will fix this.


Fix provider compatibility check crash in evp_test

EVP_MAC_CTX_get_mac_size() cannot be called on older unfixed versions before EVP_MAC_init().
Checklist

[x] tests are added or updated

ping @openssl/committers for second review

This pull request is ready to merge

Merged to the master and 3.2 branches. Thank you for the reviews.


Added archived details while archiving user

This PR is linked to Issue no. #1214: Added archived user details during the archiving process
API contract link: https://github.com/Real-Dev-Squad/website-api-contracts/pull/138
Changes Made:

added a field in db archivedDetails if the user is archived
structure:

{
  archivedDetails: {
    archived_at: Timestamp,
    archived_user_id: String,
    super_user_id: String,
    reason: String
  },
  username: String,
  github_display_name: String,
  github_id: String,
  isMember: Boolean,
  roles: {
    member: Boolean,
    in_discord: Boolean,
  },
  tokens: {
    githubAccessToken: String,
  },
  incompleteUserDetails: Boolean,
}

The reason needs to pass in the body as JSON object in the PATCH request:
**Example: **
{
    "reason":"some reason"
}

Test Case Result:

Also please add the updated API contract and model with this PR

API contract PR : https://github.com/Real-Dev-Squad/website-api-contracts/pull/138

Also please add the updated API contract and model with this PR

API contract PR : Real-Dev-Squad/website-api-contracts#138

please get review on your API contract and the data model needs to be updated in https://github.com/Real-Dev-Squad/website-data-models

hey, @heyrandhir @bhtibrewal I've added test cases, data modelling and API contacts, please review it, thanks


Why can't display date as x-axis when using the backtest_plot() function?

Hi,
I am not able to display the date as x-axis when using the "backtest_plot()" function. And I noticed that in all the tutorials, the x-axis is also empty. Is it an issue due to the matplotlib version?

Hi @aksevenli
Here https://githubmemory.com/repo/quantopian/pyfolio/issues/636 they are downgrading matplotlib to display the x-axis. Do try it out


Made the timer dropdown specify minutes

parseInt will find a number at the front of a string and get rid of everything not a number. I tested it and it works with any of the selections I tried


Introduce invariant; use it to handle display_recipient more cleanly.

This starts us using Flow's support for an assertion function named invariant, which I learned about recently: https://github.com/facebook/flow/issues/6052
The immediate inspiration is the last commit in this series, a small test cleanup which clears things up for #4327. I thought a bit about more substantial ways to make use of it, and landed on a refactoring of how we handle the notorious display_recipient property on messages: at the end of this branch, when we consume display_recipient we always do so through a pair of helpers in recipient.js, which explicitly check our assumptions and return well-typed results.
There's also a few other cleanups in here which either came up as prerequisites for the main changes, or were suggested by them and quick to add in.

This looks great! invariant reminds me of "User-defined Type Guards" in TypeScript, though I'm sure there are important differences.
I've made just a few comments, all small. It's so nice to be able to clean up display_recipient. IIUC, #3133 (among our current priorities) got stalled on cleaning up something in recipients.js; I'm not yet clear on whether it's this sort of cleanup, but I'm sure it wouldn't hurt! I happen to have just rebased #3535 (at a separate branch; see https://github.com/zulip/zulip-mobile/pull/3535#pullrequestreview-545299959), but I did so before (re)discovering the potentially preferable #4104, oops.

Merged, with that one fix. Thanks for the review!

It's so nice to be able to clean up display_recipient. IIUC, #3133 (among our current priorities) got stalled on cleaning up something in recipients.js; I'm not yet clear on whether it's this sort of cleanup, but I'm sure it wouldn't hurt! I happen to have just rebased #3535 (at a separate branch; see #3535 (review)), but I did so before (re)discovering the potentially preferable #4104, oops.

Yeah. It did stall on that when I picked it up in... February, IIRC. I then spent a solid few days studying the display_recipients and related "how to name a PM conversation" messes. A lot of that has landed since then, some is still in draft branches, and this branch overlaps with some of those but the main idea (with the two helpers to explicitly get a stream name or a PM recipients list) is something I hadn't tried before.
I'm not sure offhand how the bits that have landed compare to what was blocking #3133, or what might still be needed for that; I'd have to go take a look to page that back into my head.


refactor(containers): remove experimental bind syntax

:broken_heart: :cry: removes experimental operator in favor of es6 arrow functions—resolves #539

:+1: for your emoji!


Use integer for ts and convert it at the end

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.43%. Comparing base (c599f2f) to head (618275d).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

:exclamation:  There is a different number of reports uploaded between BASE (c599f2f) and HEAD (618275d). Click for more details.
HEAD has 37 uploads less than BASE

Flag
BASE (c599f2f)
HEAD (618275d)

38
1

Additional details and impacted files
@@             Coverage Diff             @@
##            master     #511      +/-   ##
===========================================
- Coverage   100.00%   93.43%   -6.57%
===========================================
  Files           19       19
  Lines         2161     2163       +2
===========================================
- Hits          2161     2021     -140
- Misses           0      142     +142

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


feat: add devcontainer and vscode config

This adds devcontainer and vscode configuration, which make it possible to just clone project, open in vscode devcontainer and cross compile GUI project (devcontainer is linux).

Does this actually work?

On what platform did you try that?
On windows it doesn't work for me (getting a compilation error)

This is on windows (with docker in wsl), you need to make sure you (re)open the project in the devcontainer and let it build the container and install required dotnet tooling (which it should do automatically - but may take a bit first time). Then Ctrl-Shift-B to build the GUI project. I'll just do a test to see if it works in Codespaces too

Starting in Codespaces:

Compiled inside Codespaces:

Found the solution to following error which was to dotnet clean outside of the container (it probably didn't like that there was already a build outside of it)
/usr/share/dotnet/sdk/7.0.306/Microsoft.Common.CurrentVersion.targets(4884,5): error MSB3374: The last access/last write time on file "/workspaces/ValveResourceFormat/Decompiler/obj/Debug/Decompiler.csproj.CopyComplete" cannot be set. Access to the path '/workspaces/ValveResourceFormat/Decompiler/obj/Debug/Decompiler.csproj.CopyComplete' is denied. [/workspaces/ValveResourceFormat/Decompiler/Decompiler.csproj]

But now i'm getting the following error when running VRF
A fatal error was encountered. The library 'libhostpolicy.so' required to execute the application was not found in '/workspaces/ValveResourceFormat/GUI/bin/Debug/win-x64/'

Ok, how exactly are you running it? Which project? I was actually only building inside the devcontainer and then running the resulting VRF.exe gui app from the windows host. dotnet gave me warnings that the GUI project would only run on windows so it had to be cross compiled, however I'd expect the CLI to work fine on linux, but haven't checked that ... yet

Running tests passes for me in Codespaces:

Ah, okay, you're just using the container to compile VRF, that's kind of a weird usage, why not just install .net's sdk ?
i was trying to run vrf in the container, that's what's not working, you might want to remove the launch.json if that's not usable.
I kinda don't see the usefulness of this in the repo if you can't run it inside the container

Right, the GUI app won't run in container, even less in Codespaces, decompiler does though.
Will change launch.json to run the decompiler.
For me point of running in devcontainer, is that I don't need to have any SDK stuff installed on the host (I have enough of those already) - relying on the container setup I can be sure to get all need tools in compatible version just by "run inside devcontainer".

don't think having decompiler in launch.json make sense either since it doesn't have an "interactive aspect"
i'll stick to my "it doesn't really make sense to have this merged" if doesn't allow the gui to be used (one thing i didn't try but saw mentioned was to build via the publish command dotnet publish and to target 'linux-x64', this could make it have every lib it needs)
(FYI: if this doesn't get merged and you want to hide this from git you can add it to the exclude file in .git/info/exclude)

No worries. I have in general found it helpful to add devcontainer support to projects for others to get started on it quickly, but you are right that the benefit in this case is minimal. I'll close this PR


UI shell output improvments - refer nala (apt-get alternative)

Description of the new feature / enhancement
Lately read about this apt alternative called nala and thought it would be great to have a look on some features including but not limited to parallel Installation (limited in Windows with (repackaged) msi, but not with other Installation method such as inno or Appx / msix, I believe.
Also the output of nala is notable.
Thinking about winget's role in terms of Intune app deployment nala also provides a good example on visualizing dependencies.
Source of information:
https://www-omgubuntu-co-uk.cdn.ampproject.org/v/s/www.omgubuntu.co.uk/2023/01/install-nala-on-ubuntu/
Link to Sourcecode / repository: https://gitlab.com/volian/nala
Proposed technical implementation details
Unfortunately cannot provide a concise description other than kindly asking herewith @denelon and the team to have a look and be inspired.
Goal of this PR should be looking into the implementation of nala from the point of the end user and codewise or get converted into discussion when certain ideas sprawl around this.

On Windows, generally it's not safe to install more than one ".exe" installer at a time in parallel due to the unknowns of what might happen.  MSI is also generally limited to "one at a time", and during a Windows update, the update service can block an MSI install until it has completed. MSIX has opportunity for parallel (up to six at a time I believe). Portable packages might be safe in parallel in terms of pulling binaries down and laying them on the disk, but we have to be careful about modifying the path.

The dependency view is a nice addition. We had done some early work in winget show to display dependencies, but I don't believe it ever did any form of a nested view.
I would be interested in some design ideas around that.
Package A
  + Package B
    + Package D
  + Package C

Hi @denelon, I believe your comments are dedicated to #2824 ?


docker/Debian: call latest the latest image, not testing

Using latest as a tag is a more idiomatic Docker thing to do.

This has been implemented in master.


Error in callback for watcher "config"

An error occurs when updating the page content:
[Vue warn]: Error in callback for watcher "config":
(found in <Datepicker> at mydisk:\myproject\node_modules\vue-bulma-datepicker\src\index.vue

It is caused by an error in index.vue.
There are two methods declarations, the last one overrides the first one.

There's two PR's #38 #39  fixing this

Until @luventa merges the fix, it should work on v. 1.3.0 from March 6th.

Is it possible to merge either of these PRs? It's a quick fix and it's affecting everyone.

@fundon @luventa please merge these PRs!

I have this problem too. I can confirm that the pull request solves the problem as well. Merge please :)

Yeah, please merge one of these PRs 💃

@fundon @luventa I understand it's open source, you're doing it for free and I'm in no position to demand active participation, but please just hit the damn Merge button.

Yes sir, pr has been merged and new version is just published. 😢


Error while importing pytorch_lightning

Common bugs:

Tensorboard not showing in Jupyter-notebook see issue 79.
PyTorch 1.1.0 vs 1.2.0 support see FAQ

Describe the bug
I am getting the following bug while importing pytorch_lightning

To Reproduce
Steps to reproduce the behavior:

Open terminal
type "python3"
type "import pytorch_lightning" and press "Enter"
See error

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
CUDA 10.1
python 3.7

looks like you're not in your cuda environment


Added websocket, javascript, css, and templating

Changes to finalize websocket, javascript, css, and html templating.

I like it, I am willing to merge, but I thought we were going to put the front-end in it's own repo.