Setting up multiple instances

I managed to get arkmanager working, and I'm trying to setup two instances, one running valhalla, the other the island.
I saw this post: https://github.com/FezVrasta/ark-server-tools/issues/201 which mostly makes sense, but the thing I'm curious about is why is there a /etc/arkmanager/instances/main.cfg if in that post you're setting up two arkmanager.cfg's?
Or is the instances folder a new thing since that post?  Do I just configure the arkmanager.cfg with the generic server settings, then the main.cfg & another .cfg in the instance folder for the server specific settings?
Also how do port allocations go for additional instances on the same machine?

I saw this post: #201 which mostly makes sense, but the thing I'm curious about is why is there a /etc/arkmanager/instances/main.cfg if in that post you're setting up two arkmanager.cfg's?
Or is the instances folder a new thing since that post?

The instances directory is a new feature that was added some time after that issue was closed.  Since then, the instance config has been completely split out to a separate config file, with only common settings being stored in arkmanager.cfg.

Do I just configure the arkmanager.cfg with the generic server settings, then the main.cfg & another .cfg in the instance folder for the server specific settings?

Yes.  If you don't want a "main" instance, then its config file can be deleted, and the defaultinstance="main" line can be removed from arkmanager.cfg.  It is there as a convenience for most users who only want one instance.

Also how do port allocations go for additional instances on the same machine?

You need to configure the ports in the instance configs, ensuring the different instances have different ports, and ensure each instance runs in its own arkserverroot.

Thanks for that.
Removing the defaultinstance would just mean I'd have to specify an instance every time right?  If I leave it there, then the main one I wouldn't specify, and the secondary I'd have to specify?
When specifying an alternate config at the bottom of arkmanager.cfg the name I list as configfile_ means when I use arkmanager I don't have to use the full file path, just the name right?
So configuring it like this is good/bad?
/etc/arkmanager/arkmanager.cfg
arkstChannel="master"                                               # change it to a different branch to get non-stable versions
install_bindir="/usr/local/bin"
install_libexecdir="/usr/local/libexec/arkmanager"
install_datadir="/usr/local/share/arkmanager"

# config SteamCMD
steamcmdroot="/home/steam/steamcmd"                                 # path of your steamcmd instance
steamcmdexec="steamcmd.sh"                                          # name of steamcmd executable
steamcmd_user="steam"                                               # name of the system user who own steamcmd folder
steamcmd_appinfocache="/home/steam/Steam/appcache/appinfo.vdf"      # cache of the appinfo command
#steamlogin="anonymous"                                             # Uncomment this to specify steam login instead of using anonymous login

# config environment
arkserverexec="ShooterGame/Binaries/Linux/ShooterGameServer"        # name of ARK server executable
arkbackupdir="/home/steam/ARK-Backups"                              # path to backup directory
arkwarnminutes="5"                                                 # number of minutes to warn players when using update --warn
arkautorestartfile="ShooterGame/Saved/.autorestart"                 # path to autorestart file
arkAutoUpdateOnStart="false"                                        # set this to true if you want to always update before startup
arkBackupPreUpdate="true"                                          # set this to true if you want to perform a backup before updating
arkStagingDir="/home/steam/ARK-Staging"                            # Uncomment to enable updates to be fully downloaded before restarting the server (reduces downtime while updating)

# Options to automatically remove old backups to keep backup size in check
# Each compressed backup is generally about 1-2MB in size.
arkMaxBackupSizeMB="500"                                            # Set to automatically remove old backups when size exceeds this limit
#arkMaxBackupSizeGB="2"                                             # Uncomment this and comment the above to specify the limit in whole GB

# Update warning messages
# Modify as desired, putting the %d replacement operator where the number belongs
msgWarnUpdateMinutes="This ARK server will shutdown for an update in %d minutes"
msgWarnUpdateSeconds="This ARK server will shutdown for an update in %d seconds"
msgWarnRestartMinutes="This ARK server will shutdown for a restart in %d minutes"
msgWarnRestartSeconds="This ARK server will shutdown for a restart in %d seconds"
msgWarnShutdownMinutes="This ARK server will shutdown in %d minutes"
msgWarnShutdownSeconds="This ARK server will shutdown in %d seconds"

# ARK server common options - use ark_<optionname>=<value>
# comment out these values if you want to define them
# inside your GameUserSettings.ini file
#serverMap=""                                               # server map (default TheIsland)
#serverMapModId="504122600"                                         # Uncomment this to specify the Map Mod Id (<fileid> in http://steamcommunity.com/sharedfiles/filedetails/?id=<fileid>)
#ark_TotalConversionMod="496735411"                                 # Uncomment this to specify a total-conversion mod
ark_RCONEnabled="True"                                              # Enable RCON Protocol
ark_ServerPassword="password"                                               # ARK server password, empty: no password required to login
ark_ServerAdminPassword="adminpassword"                               # ARK server admin password, KEEP IT SAFE!
ark_MaxPlayers="10"
#ark_GameModIds="487516323,487516324,487516325"                     # Uncomment to specify additional mods by Mod Id separated by commas
#ark_AltSaveDirectoryName="SotF"                                    # Uncomment to specify a different save directory name

# ARK server flags - use arkflag_<optionname>=true
#arkflag_OnlyAdminRejoinAsSpectator=true                            # Uncomment to only allow admins to rejoin as spectator
#arkflag_DisableDeathSpectator=true                                 # Uncomment to disable players from becoming spectators when they die

# ARK server options - i.e. for -optname=val, use arkopt_optname=val
#arkopt_StructureDestructionTag=DestroySwampSnowStructures

# config Service
logdir="/var/log/arktools"                                          # Logs path (default /var/log/arktools)

# steamdb specific
appid=376030                                                        # Linux server App ID
mod_appid=346110                                                    # App ID for mods

# Mod OS Selection
mod_branch=Windows
# Add mod-specific OS selection below:
#mod_branch_496735411=Windows

# alternate configs
# example for config name "ark1":
#configfile_ark1="/path/to/config/file"
configfile_TheIsland=/etc/arkmanager/instances/TheIsland.cfg
configfile_Valhalla=/etc/arkmanager/instances/Valhalla.cfg

#defaultinstance="main"

Now /etc/arkmanager/instances/Valhalla.cfg
# config environment
arkserverroot="/home/steam/Valhalla"                                     # path of your ARK server files (default ~/ARK)

# ARK server options - use ark_<optionname>=<value>
# comment out these values if you want to define them
# inside your GameUserSettings.ini file
serverMap="Valhalla"                                               # server map (default TheIsland)
serverMapModId="504122600"                                         # Uncomment this to specify the Map Mod Id (<fileid> in http://steamcommunity.com/sharedfiles/filedetails/?id=<fileid>)
#ark_TotalConversionMod="496735411"                                 # Uncomment this to specify a total-conversion mod
ark_RCONEnabled="True"                                              # Enable RCON Protocol
ark_RCONPort="32330"                                                # RCON Port
ark_SessionName="Valhalla"                                  # if your session name needs special characters please use the .ini instead
ark_Port="7779"                                                     # ARK server port (default 7778)
ark_QueryPort="28015"                                               # ARK query port (default 27015)
#ark_ServerPassword=""                                               # ARK server password, empty: no password required to login
#ark_ServerAdminPassword=""                               # ARK server admin password, KEEP IT SAFE!
#ark_MaxPlayers="70"
#ark_GameModIds="487516323,487516324,487516325"                     # Uncomment to specify additional mods by Mod Id separated by commas
#ark_AltSaveDirectoryName="SotF"                                    # Uncomment to specify a different save directory name

# ARK server flags - use arkflag_<optionname>=true
#arkflag_OnlyAdminRejoinAsSpectator=true                            # Uncomment to only allow admins to rejoin as spectator
#arkflag_DisableDeathSpectator=true                                 # Uncomment to disable players from becoming spectators when they die

# ARK server options - i.e. for -optname=val, use arkopt_optname=val
#arkopt_StructureDestructionTag=DestroySwampSnowStructures

and lastly /etc/arkmanager/instances/TheIsland.cfg
# config environment
arkserverroot="/home/steam/TheIsland"                                     # path of your ARK server files (default ~/ARK)

# ARK server options - use ark_<optionname>=<value>
# comment out these values if you want to define them
# inside your GameUserSettings.ini file
serverMap="TheIsland"                                               # server map (default TheIsland)
#serverMapModId="504122600"                                         # Uncomment this to specify the Map Mod Id (<fileid> in http://steamcommunity.com/sharedfiles/filedetails/?id=<fileid>)
#ark_TotalConversionMod="496735411"                                 # Uncomment this to specify a total-conversion mod
ark_RCONEnabled="True"                                              # Enable RCON Protocol
ark_RCONPort="32331"                                                # RCON Port
ark_SessionName="TheIsland"                                  # if your session name needs special characters please use the .ini instead
ark_Port="7780"                                                     # ARK server port (default 7778)
ark_QueryPort="28016"                                               # ARK query port (default 27015)
#ark_ServerPassword=""                                               # ARK server password, empty: no password required to login
#ark_ServerAdminPassword=""                               # ARK server admin password, KEEP IT SAFE!
#ark_MaxPlayers="70"
#ark_GameModIds="487516323,487516324,487516325"                     # Uncomment to specify additional mods by Mod Id separated by commas
#ark_AltSaveDirectoryName="SotF"                                    # Uncomment to specify a different save directory name

# ARK server flags - use arkflag_<optionname>=true
#arkflag_OnlyAdminRejoinAsSpectator=true                            # Uncomment to only allow admins to rejoin as spectator
#arkflag_DisableDeathSpectator=true                                 # Uncomment to disable players from becoming spectators when they die

# ARK server options - i.e. for -optname=val, use arkopt_optname=val
#arkopt_StructureDestructionTag=DestroySwampSnowStructures

So if I'm interpreting it correctly, I've got two copies of the server at /home/steam/TheIsland & /home/steam/Valhalla.
The arkmanager.cfg is setting both servers to have a password of "password", admin password is "adminpassword", max players of 10 & setting taming speed, difficulty, gamma settings, third person & notifications of players joining and leaving.
Then the seperate instance cfg's are setting the ports for RCON, QueryPort & Port aswell as the names & for valhalla the map mod ID.
Then to administer each of the servers, I just type arkmanager TheIsland or Valhalla then the command?
One thing I've not been able to find documentation on is how to set the servers to auto start with the machine booting up?  This is on ubuntu 14.04, so using upstart not systemd.

Removing the defaultinstance would just mean I'd have to specify an instance every time right? If I leave it there, then the main one I wouldn't specify, and the secondary I'd have to specify?

The defaultinstance setting just sets which instance to execute a command on if no instance is specified.  Without the defaultinstance setting, yes you need to specify which instance to execute the command on, as otherwise it will ask if you want to execute the command on all instances.

When specifying an alternate config at the bottom of arkmanager.cfg the name I list as configfile_ means when I use arkmanager I don't have to use the full file path, just the name right?

If the instance configs are in /etc/arkmanager/instances, then you don't need the configfile_xxx=... settings.  If you do specify the configfile_xxx=... settings, these must each specify the full path to the instance config file.

configs

Until #438 is merged, you probably want to change serverMapModId="504122600" to ark_MapModID="504122600".

So if I'm interpreting it correctly, I've got two copies of the server at /home/steam/TheIsland & /home/steam/Valhalla.

Yes.  However, the files outside ShooterGame/Saved on one instance can be hard-linked to the files outside ShooterGame/Saved in the other instance in order to save disk space.

The arkmanager.cfg is setting both servers to have a password of "password", admin password is "adminpassword", max players of 10 & setting taming speed, difficulty, gamma settings, third person & notifications of players joining and leaving.

The first 3 are correct.  The remainder can be specified as either options in arkmanager.cfg or the instance config (in which case they are passed on the ShooterGameServer comand-line) or in the server's GameUserSettings.ini or Game.ini files

Then the seperate instance cfg's are setting the ports for RCON, QueryPort & Port aswell as the names & for valhalla the map mod ID.

Correct.

Then to administer each of the servers, I just type arkmanager TheIsland or Valhalla then the command?

You type arkmanager <command> @<instance> <args> where <command> is the command you want to run, <instance> is the instance you want to run the command on, and <args> are the arguments to the command.
e.g. to broadcast "Hello everybody" to everyone on TheIsland, run arkmanager broadcast @TheIsland "Hello everybody"
To save the world on both instances at the same time, run arkmanager saveworld @TheIsland @Valhalla or arkmanager saveworld @all

One thing I've not been able to find documentation on is how to set the servers to auto start with the machine booting up? This is on ubuntu 14.04, so using upstart not systemd.

The arkmanager init script installed in /etc/init.d should work.
If Ubuntu 14.04 doesn't run the init scripts in /etc/init.d, or if you'd like to use the upstart init scripts, then you should be able to copy the tools/upstart/arkmanager.conf and tools/upstart/arkmanager-instance.conf files into /etc/init, and remove /etc/init.d/arkmanager.

ok so now using those configs, I'm getting the valhalla server show up, but not the island on lan or unofficial servers (I'm showing ones with passwords set).
Arkmanager is giving me arkservers.net links that are picking up both servers, arkmanager says both are listening and netstat shows the 6 ports active (query, port, rcon).
Any ideas what I should look at to figure out why the island instance isn't showing on the server list ingame?

I myself have to refresh the server list in-game a few times before the server on my LAN will show up.  I think it's a limitation of the ARK client's server list.
It also seems to take a short time between when the ARK server sets its max player count to non-zero and when it's actually ready to accept connections.

I'm leaving it atleast 10 minutes before checking, and now after restarting the server the servers are loading, listening on the queryport, but not the other port.  arkmanager status @all is reporting both instances to be running, not listening & reporting the server version 1003475.
Any ideas?  I'm figuring there's no point refreshing the server list at this point if it's not opening both of the ports required now.

Ok, so after tooling around some more (using the same configuration as above) the server seems to be loading two copies of valhalla, with the same server name.
I managed to test this by connecting directly without the server browser ingame using this method
Connecting to each port, it seems to be loading both instances, but for some reason the instance playing on TheIsland is loading with valhalla as the map and the same name as the valhalla server.
I've had a look at the gameusersettings.ini in /home/steam/TheIsland/Shootergame/Saved/Config/LinuxServer/ & it's showing the correct name.
Any ideas as to why or where I'd look?  If I can't resolve it before tonight, I think I'll delete both instances of the server software & run arkmanager install @all tonight to start fresh.
Full disclosure, I initially downloaded one instance, installed the valhalla mod using arkmanager & then when I realised I could run two instances I made a copy of the server containing the valhalla mod (if this helps diagnose it).

arkmanager logs the command that it uses to start the server in /var/log/arktools/arkserver.log
Please compare the command logged for Valhalla with that logged for TheIsland.
With the second server (TheIsland), since you copied the server from one directory to another, and since you haven't had any success yet, it may be necessary to remove the contents of the ShooterGame/Saved directory.

This is an excerpt of the arkserver.log when I ran them.
17:52:52: Running /home/steam/Valhalla/ShooterGame/Binaries/Linux/ShooterGameServer Valhalla\?AllowThirdPersonPlayer=True\?DifficultyOffset=1.0\?EnablePVEGamma=True\?MapModId=504122600\?MaxPlayers=10\?Port=7778\?QueryPort=28015\?RCONEnabled=True\?RCONPort=32330\?ServerAdminPassword=adminpassword\?ServerPassword=password\?SessionName=DamageInc\ Valhalla\?TamingSpeedMultiplier=4\?alwaysNotifyPlayerJoined=True\?alwaysNotifyPlayerLeft=True\?listen
17:52:52: Running /home/steam/TheIsland/ShooterGame/Binaries/Linux/ShooterGameServer TheIsland\?AllowThirdPersonPlayer=True\?DifficultyOffset=1.0\?EnablePVEGamma=True\?MaxPlayers=10\?Port=7779\?QueryPort=28016\?RCONEnabled=True\?RCONPort=32331\?ServerAdminPassword=adminpassword\?ServerPassword=password\?SessionName=DamageInc\ TheIsland\?TamingSpeedMultiplier=4\?alwaysNotifyPlayerJoined=True\?alwaysNotifyPlayerLeft=True\?listen
17:52:52: Server PID: 7654
17:52:52: Server PID: 7655
[S_API FAIL] SteamAPI_Init() failed; SteamAPI_IsSteamRunning() failed.
[S_API FAIL] SteamAPI_Init() failed; SteamAPI_IsSteamRunning() failed.
Setting breakpad minidump AppID = 346110
Setting breakpad minidump AppID = 346110
17:53:50: server is up
17:54:21: server is up

Unless there's something I'm missing, it seems to be executing what it should be, same result after deleting the contents of /home/steam/TheIsland/ShooterGame/Saved/ directory.
Any other ideas other than delete the server files completely and reinstall?

Only other suggestions I can give are to put more separation between ARK ports - e.g. when testing multiserver, I have been putting a 10 port gap between servers (i.e. server1.Port=7778, server2.Port=7788, etc, and doing the same for QueryPort).

So... I figured out the issue, and it wasn't you guys.
My router ui has a bug in it that caused the range redirect I setup to funnel 28015-16 into 28015 instead of 28015-28016...
Really appreciate you sticking with me :) Thanks again for the help!

Arg, more questions if you're willing.  You mentioned "Yes. However, the files outside ShooterGame/Saved on one instance can be hard-linked to the files outside ShooterGame/Saved in the other instance in order to save disk space.", wouldn't this give file in use errors?  I'm guessing this is making the system think it has two copies, but the 2nd is just links to the initial one?
Also when running backups, it's just naming them date_time.  Is there any plans to include some instance identifier for them?  Otherwise if I need to restore a particular server, I am going to have to wade through backups, restoring them to find the one I'm after & testing by logging in to make sure it's the right map.


external ML-API

trying to get this setup with the Obico ML-API running in docker on a seperate device than my home assistant instance (accessible via the network). Is there anything I need to change with the URL? I am using http://my-ml-api-container-serverip:3333/hc which prints "ok" when I go to it with my browser so I believe the API is accessible. When I turn the Obico ML Communication switch on it becomes unavailable. The Obico ML Detection Camera also shows a broken thumbnail image.
Any ideas?

Can you post HA logs and Obico ML API server logs?

I should be able to get full logs to share this afternoon,
In the HA logs this line leads me to believe my API url I'm using may not be correct.
_File "/config/custom_components/obico_ml/coordinator.py", line 54, in async_update_data
raise Exception(f"API call failed: {detection_response.status}")
Exception: API call failed: 404

If you're using vanilla Obico ML API container and not my fork (https://github.com/nobodyguy/obico_ml_ha_addon), it's not currently supported.

Thank you, I appreciate the help, I will look into some alternative options, I have an x86 architecture machine on the way so that should open up my options some.

Btw you can also run my addon's container as a regular docker container if you want.

I was trying that actually but was getting the error :
/init: 45: exec: s6-overlay-suexec: Exec format error
This is an on oracle cloud server with an arm cpu and running Ubuntu/docker. any ideas?

Is the droidik/obico_ml_ha_addon-aarch64 image used instead of amd64?
I can also imagine that the image can be built incorrectly.

I did try to change from AMD64 to aarch64 and also removed the AMD64 option below that. I was thinking I may need to rebuild the image possibly in relation to what you mentioned or as you mentioned


add reflect app template

Docs Build status updates of commit 0eab1d0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/assets/images/reflect-app-user-poll.png
:white_check_mark:Succeeded
View

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit e87cac9:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/assets/images/reflect-app-user-poll.png
:white_check_mark:Succeeded
View

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit c0a5b46:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/assets/images/reflect-app-user-poll.png
:white_check_mark:Succeeded
View

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit fd327c7:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/assets/images/reflect-app-user-poll.png
:white_check_mark:Succeeded
View

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Git checkout failed on shallow repository, this might because of git fetch with depth '1' doesn't include the checkout commit

Hello
I added Agent.Source.Git.ShallowFetchDepth to pipeline:
variables:
  Agent.Source.Git.ShallowFetchDepth: 100

and fetchDepth to steps:
  - checkout: self
    fetchDepth: 100

And even here:

But still get "FetchDepth":"1":
##[debug]Processed: ##vso[telemetry.publish area=AzurePipelinesAgent;feature=GitFetch]{"ElapsedTimeMilliseconds":"6642","RefSpec":"","RemoteName":"origin",**"FetchDepth":"1"**,"ExitCode":"0","Options":"--force --tags --prune --prune-tags --progress --no-recurse-submodules origin **--depth=1** "}
Git checkout failed on shallow repository, this might because of git fetch with depth '1' doesn't include the checkout commit
Question: There are any other places where FetchDepth is 1?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 800c5477-b39f-48e6-6f9c-aa7994b2ec2a
Version Independent ID: db9e2e97-d44d-734e-6a6f-b01f60f01c95
Content: Options for Git repositories - Azure Pipelines
Content Source: docs/pipelines/repos/pipeline-options-for-git.md
Service: azure-devops-pipelines
Sub-service: azure-devops-pipelines-repos
GitHub Login: @steved0x
Microsoft Alias: sdanie

@mpushki -- Maria, thank you for your thorough feedback and question.  Please consider these resources:

Azure DevOps Services
Azure DevOps on Stack Overflow

@steved0x -- Steve, please look into this potential documentation issue.

Hi @WilliamAntonRohm, I'm not sure that it is doc-bug.
It should work right?
  - checkout: self
    fetchDepth: 100

@mpushki -- Maria, thank you for following up.  Your question is outside my expertise.  If you feel this is a product issue, please add it here:

Developer Community

@davidstaheli -- David, can you shed some light on this?

Also, I found only one place where it works for me. But it still does not work in other places


Permission problems

It looks like your docker image defaults to uid 1000, gid 1000, which is a problem for me. Due to a lot of history, my uid is 500 and the user gid is 100. I would like the docker image to run with my uid (500) and with gid 2000, which I have assigned for use by crashplan since I have several TB of data backed up locally.
Right now there are a lot of permission issues. It cannot access much of my data (due to a mismatching uid) and cannot access the backup medium as well.
Is there any way to assign it to use these IDs?

Yes, you need to adjust the USER_ID and GROUP_ID environment variables.
See:
https://github.com/jlesage/docker-crashplan-pro#environment-variables
https://github.com/jlesage/docker-crashplan-pro#usergroup-ids

Thanks.  It's now working and able to synchronize with my local backup drive. I created a new group for this.

Great!


Split up the Feed/Draper adapters

[x] Make the FeedAdapter work independently (when feedUrl+episodeGuid params are present)
[x] Add a new DraperAdapter (when feedId+episodeGuid params are present)
[x] Tests!

Running in dev, you can see a regular feed here, and a draper feed here.

off the top of my head, the functionality that we're losing (that we're actually taking advantage of) is using the most recent episode when an episode guid is not provided.

Going to merge this now, and revisit the "latest episode" in another PR.


stream core module

Is it possible to use the stream module in this version of nginx  ?
https://nginx.org/en/docs/stream/ngx_stream_core_module.html

You should be able to use any nginx image using the separate container model.  I haven't tried it, but can't think of a reason why it would not work.


Changed up.go to run pre-up tasks before any loading

Resolves #805
Moved the pre-up tasks to run before any environment loading, but after setting vars and loading the logs.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: sam-doordash sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Closing in favor of #813


PDF details report section Step/Hook details returns the execution steps

Hi,
In PDF Extent report, i have set the displayDetailed: true in pdf-config.yaml file.
Once the PDF Extent report generated under the details section->for any particular scenario the Step/Hook Details it returns the
steps say e.g. below
Given I register a user
When I send a post a request
Then login should be successful.
My query is along with the above step it also returns the step execution output e.g. Request headers, Response details etc.
Is there any way to display only the step not the step execution output?
Thanks.

Currently this is not supported. What is the reason you desire this output?
Thanks

Currently this is not supported. What is the reason you desire this output?
Thanks

Hello,
The pdf report generated is very huge (due to multiple steps for a particular scenario in test suite ) and secondly the graph generated for each scenario and details around is taking quite space which i feel is not much helpful.


Fix postinstall script

I was trying to install browserless in Windows, but installation failed me, due to  this package's postinstall script.
curl is not available by default in Windows and it's not allowed to exec scripts  in this way  ./scripts/postinstall.
In addition, as weekend kata for me I made faster the denormalization code block.

And, I noticed that the json from disconnectme have some fields with strings like "true" or "eef" and on the result the strings was unfolded. I avoid that.
{
 "Adless": {
  "https://www.adless.io/": [
   "adless.io"
  ],
  "performance": "true",
  "cryptominer": "true"
  }
}

[
(...)
"t",
"r",
"u",
"e"
(...)
]

Pull Request Test Coverage Report for Build 27

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 24:
0.0%

Covered Lines:
4

Relevant Lines:
4

💛  - Coveralls

Awesome, just a few of comments, check it 🙂

Released at 1.1.3 🎉


refactor: refactor implementation

The code is kind of messy ATM since I didn't know how to implement the API at first, but now that it's got a decently finished implementation, this PR will fix that.

Codecov Report
Base: 99.66% // Head: 99.67% // Increases project coverage by +0.01% :tada:

Coverage data is based on head (4ebb1a2) compared to base (d58233b).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##               main      #12      +/-   ##
============================================
+ Coverage     99.66%   99.67%   +0.01%
- Complexity      104      110       +6
============================================
  Files            19       20       +1
  Lines           295      307      +12
  Branches         39       39
============================================
+ Hits            294      306      +12
  Partials          1        1

Impacted Files
Coverage Δ

...java/net/jailgens/enchanted/MirrorCommandInfo.java
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Set bot icon

Right now there's sad, grey-coloured robot. PropsApp bot should not be sad at all.

LGTM :)


Add MongoDB replica set example

@bparees @jhadvig @danmcp this is a new PR that use one-shot member management Pod that will initialize the replica.

@mfojtik we're using tabs in shell scripts in our images

@mnagy then we are doing it wrong :-)

@mfojtik +1

@mnagy configure your vim with :set expandtab and then go through all scripts we have and write :retab :-)

I made some refactoring to the names of Pod(s) and containers to make it more clear and to be more descriptive about their roles in the cluster. Also renamed the 'replica_manager.sh' to 'initiate_replica.sh' as the Pod is on-shot and it does not really manage replica Pods.

@danmcp @jhadvig PTAL (this is now ready for merge)

@danmcp fixed, thanks!

[merge]

@mfojtik what about tests ?


Do not deviate from UBSan default for local fuzzing

OSS-Fuzz uses different compile-time settings for UBSan tuned for their setup, but it doesn't make sense to enforce these also for local users.
In OSS-Fuzz mode, these flags will still be picked up from the environment.

Waiting for all checks to pass before merging.

@stefanbucur Tests pass after the rebase.


chore: Simplify CometShuffleMemoryAllocator to use Spark unified memory allocator

Which issue does this PR close?

Closes #1064.
Rationale for this change

What changes are included in this PR?

How are these changes tested?

test failure:
 [info] - Spark vectorized reader - with partition data column - select a single complex field from a map entry and its parent map entry *** FAILED *** (653 milliseconds)
[info]   org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 215.0 failed 1 times, most recent failure: Lost task 0.0 in stage 215.0 (TID 370) (4bf8ef4698e6 executor driver): java.lang.IllegalArgumentException: CometShuffleMemoryAllocator should be used with off-heap memory mode, but got ON_HEAP
[info] 	at org.apache.spark.shuffle.comet.CometShuffleMemoryAllocator.getInstance(CometShuffleMemoryAllocator.java:44)
[info] 	at org.apache.spark.sql.comet.execution.shuffle.CometDiskBlockWriter.<init>(CometDiskBlockWriter.java:139)
[info] 	at org.apache.spark.sql.comet.execution.shuffle.CometBypassMergeSortShuffleWriter.write(CometBypassMergeSortShuffleWriter.java:181)

I think we need to specify spark.memory.offHeap.enabled=true when running Spark tests? I need to do the same in https://github.com/apache/datafusion-comet/pulls
For this PR we should also fall back to Spark for shuffle if spark.memory.offHeap.enabled=false?

Basically Spark tests are running with on-heap config, except for tests that particularly for off-heap test.
I'm not sure if enabling off-heap for all Spark tests can pass them all. If it works, let's do it.
If not, I plan to keep and rename current CometShuffleMemoryAllocator to a test-only class CometTestShuffleMemoryAllocator. Once it runs Spark tests, Comet can use CometTestShuffleMemoryAllocator to run Spark tests.

@andygrove All Spark tests are passed now.

I tried testing with TPC-H but see a memory issue:
│ 24/11/08 02:31:44 INFO core/src/lib.rs: Comet native library version 0.4.0 initialized                                                                                │
│ #                                                                                │
│ # A fatal error has been detected by the Java Runtime Environment:                                                                                │
│ #                                                                                │
│ #  SIGSEGV (0xb) at pc=0x00007399e4661564, pid=11, tid=132                                                                                │
│ #                                                                                │
│ # JRE version: OpenJDK Runtime Environment Temurin-11.0.24+8 (11.0.24+8) (build 11.0.24+8)                                                                                │
│ # Java VM: OpenJDK 64-Bit Server VM Temurin-11.0.24+8 (11.0.24+8, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)                                                                           │
│ # Problematic frame:                                                                                │
│ corrupted double-linked list

One other issue. I tested with spark.memory.offHeap.enabled=false and the shuffle did not fall back to Spark but failed at runtime.

I tried testing with TPC-H but see a memory issue:

I will test it locally too.

One other issue. I tested with spark.memory.offHeap.enabled=false and the shuffle did not fall back to Spark but failed at runtime.

Yes. If using on-heap config, CometShuffleMemoryAllocator will throw runtime error.
The test only CometTestShuffleMemoryAllocator is only used in Spark/Comet tests.

I tried testing with TPC-H but see a memory issue:

Hmm, I just ran TPC-H with this PR on Spark 3.4 using datafusion-comet script without any error.

Yes. If using on-heap config, CometShuffleMemoryAllocator will throw runtime error, i.e., you need to use off-heap config in Spark.

Right, so if the user is using on-heap, we should not use Comet shuffle and should fall back to Spark. We probably just need to update isCometShuffleEnabled to check if off-heap is being used.

Yes. If using on-heap config, CometShuffleMemoryAllocator will throw runtime error, i.e., you need to use off-heap config in Spark.

Right, so if the user is using on-heap, we should not use Comet shuffle and should fall back to Spark. We probably just need to update isCometShuffleEnabled to check if off-heap is being used.

Oh, I see. That sounds good. I will update it.

I don't set spark.comet.memory.overhead.factor. Do you need it?

Actually, this PR won't close #886 because this is still using a singleton, so let's ignore that for now.
This PR LGTM and I will approve after some more testing.

It fallbacks to Spark shuffle now if off-heap is not enabled.

Codecov Report
Attention: Patch coverage is 59.09091% with 36 lines in your changes missing coverage. Please review.

Project coverage is 34.19%. Comparing base (845b654) to head (e7e7847).
Report is 13 commits behind head on main.

Files with missing lines
Patch %
Lines

...shuffle/comet/CometTestShuffleMemoryAllocator.java
68.96%
9 Missing and 9 partials :warning:

...ark/shuffle/comet/CometShuffleMemoryAllocator.java
30.76%
8 Missing and 1 partial :warning:

.../comet/execution/shuffle/CometDiskBlockWriter.java
0.00%
4 Missing :warning:

...org/apache/comet/CometSparkSessionExtensions.scala
20.00%
0 Missing and 4 partials :warning:

...spark/sql/comet/execution/shuffle/SpillWriter.java
50.00%
1 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1063      +/-   ##
============================================
- Coverage     34.46%   34.19%   -0.28%
+ Complexity      888      884       -4
============================================
  Files           113      115       +2
  Lines         43580    42765     -815
  Branches       9658     9346     -312
============================================
- Hits          15021    14622     -399
+ Misses        25507    25279     -228
+ Partials       3052     2864     -188

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Actually, this PR won't close #886 because this is still using a singleton, so let's ignore that for now.

As now the allocator uses all available memory on the executor (we don't specify memory size on the allocator), it should not be an issue for #886 now. @andygrove Do you want to re-check if #886 can be fixed by this PR too? Thanks.

Can we make COMET_COLUMNAR_SHUFFLE_MEMORY_SIZE and COMET_COLUMNAR_SHUFFLE_MEMORY_FACTOR internal configs now, since they are only used in tests now?

Can we make COMET_COLUMNAR_SHUFFLE_MEMORY_SIZE and COMET_COLUMNAR_SHUFFLE_MEMORY_FACTOR internal configs now, since they are only used in tests now?

Yes. They should be internal configs now.

As now the allocator uses all available memory on the executor (we don't specify memory size on the allocator), it should not be an issue for #886 now. @andygrove Do you want to re-check if #886 can be fixed by this PR too? Thanks.

I will test this again today.

I'm running into SIGSEGV issues again.
│ # A fatal error has been detected by the Java Runtime Environment:                                                                                │
│ #                                                                                │
│ #  SIGSEGV (0xb) at pc=0x000072e2c93b6bc8, pid=11, tid=127                                                                                │
│ #                                                                                │
│ # JRE version: OpenJDK Runtime Environment Temurin-11.0.24+8 (11.0.24+8) (build 11.0.24+8)                                                                                │
│ # Java VM: OpenJDK 64-Bit Server VM Temurin-11.0.24+8 (11.0.24+8, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)                                                                     │
│ # Problematic frame:                                                                                │
│ # C  [libcomet-14210005976568904946.so+0x736bc8]  comet::execution::shuffle::row::append_columns::h9b53b563e484a30e+0x1318                                                                              │
│ #

I will try running the same benchmark on main.

I increased the off-heap pool size, and now I can run TPC-H q5 @ sf=1TB on the main branch, but get SIGSEGV with this PR.

Let me see if I can reproduce it.

Ah, I figured out what was wrong there. I updated this with the change.
Please also run the benchmarks to verify it. Thanks. @andygrove

Cool. Thanks @andygrove for verifying it.


tests: irq_lock: add test to validate irq_lock

The irq_lock is thread-specific api. With the use of irq_lock api
thread can temporarily prevent all IRQ handling in the system.
Signed-off-by: Ajay Kishore<EMAIL_ADDRESS>

Codecov Report

Merging #6590 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6590   +/-   ##
=======================================
  Coverage   53.46%   53.46%
=======================================
  Files         437      437
  Lines       41391    41391
  Branches     7934     7934
=======================================
  Hits        22128    22128
  Misses      16043    16043
  Partials     3220     3220

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b2dfb32...b0d0bcb. Read the comment docs.

@nashif , @AdithyaBaglody : Addressed your comment and pushed new patch. Please have a look it.

recheck

@nashif : Addressed your comment and pushed a new patch. Please have a look it.

@nashif after adding ztest_test_skip() shippable is failing, can you have a look once?

please put all irq related tests into one test project, no reason to have a new project for every single API tests.

@nashif moved this test under a common "interrupt" folder.

@nashif moved this test under a common "interrupt" folder. Could you please have a look.

Can this be grouped with nested_irq test?

is this PR still relevant?

@nashif No its not relevant. I have already clubbed together these changes with PR 6529.


Test Teensy operation without USB_VIN

I'm not sure how to connect V_USB_IN (fused, dioded supply from USB) to the Teensy/MicroMod USB_VIN pin.
Teensy uses USB_VIN for power and I'm not sure that USB communication/programming will work without it. But that would kind of defeat the point of disconnecting USB power.
At the very least, I need to move the connection to behind the diode and make a solderable disconnect, possibly a jumper...

Conflicting information here. From the schematics, it seems like V_USB isn't necessary to run. But it seems (https://forum.pjrc.com/index.php?threads/teensy-4-1-powered-from-3-3v-pin-not-initializing-usb-communication.73613/) like it might be necessary for it to have 3.3V or 5V or nothing.
Resources

https://www.pjrc.com/store/ic_mkl02_t4.html

Takeaways

V_BATT should be powered first. Might need to use the coin cell even if not using the RTC

To cover my ass: Connect V_USB_IN (pre-diode), 3.3V, and possibly V+ behind diodes and a solder/real jumper to USB_VIN on the Teensy

https://forum.pjrc.com/index.php?threads/powering-micromod-from-3-3v.68305/ is a good account of what happens with MM

From https://forum.pjrc.com/index.php?threads/powering-micromod-from-3-3v.68305/:

I removed the F2 fuse and left BYP open on the ATP carrier. I connected external power (3.3V) to V_IN on the ATP.
I left the MEAS jumper in place on the Teensy Micromod
I have a Coin Cell installed on the ATP (if that matters).


⭐ Add a CONTRIBUTING.md to Guide New Contributors

🎤 Tell us your idea
Issue
Add a CONTRIBUTING.md file to the CopilotKit repository.
Expected Impact:
Adding a CONTRIBUTING.md file will help guide new contributors by providing clear instructions on how to get involved. This can enhance community engagement and streamline the contribution process.
Suggested Content:

Guidelines for setting up the development environment
Steps for creating and submitting a pull request

Example:
https://github.com/pezzolabs/pezzo/blob/main/CONTRIBUTING.md

Sir can you please assign me this issue instead?

Have raised the PR Sir #662
https://github.com/CopilotKit/CopilotKit/pull/662

Can you assign this to me? @NathanTarbert

hey, I would like to work on this issue please assign it to me.

Please check the PR #680 Sir @NathanTarbert


BooruBot v4: Rewriting the entirety of BooruBot

Ah shit, here we go again
Shoutout to the previous rewrite (v3) nearly 6 years ago #14
Moving to Typescript, a module system that's actually on npm and not just copy/pasted, interactions (!!!), and much much much more stable code

† -> Likely requires significant effort
? -> Might not be implemented
→ -> Renamed
Database

[x] Setup Prisma
[x] Migrate Database†

Commands

[x] Blacklist → /blacklist (add|remove|view|clear) (tag|site) & /view blacklist

[ ] More advanced blacklist (combine tags, etc)† later
/blacklist view for mods, /view blacklist for users

Discord only lets you assign perms to the root/base command (blacklist/view), which is used to manage access to who can edit the blacklist.

[x] delsettings → /config delete
[x] Settings → /config
[x] Link
[x] Search† → /search
[x] Sites → /view sites

Support Server - Privacy Policy - Terms of Service
What changed?
🎉 - New
⚠ - Changed
❌ - Removed

Slash commands instead of message commands (Why?)

⚠ Message commands don't exist at all anymore (Besides pinging the bot for some helpful links)
⚠ The new commands are:

b!search (or b!s, b!sb, b!r...) → /search
b!link → /link
b!settings (or b!set...) → /config
b!delsettings → /config delete
b!sites → /view sites
Blacklisting:

b!blacklist tag|site (or b!bl) → /blacklist add tag|site
b!whitelist tag|site (or b!wl) → /blacklist remove tag|site
b!blacklist (view blacklist) → /blacklist view (mods) or /view blacklist (everyone)

Misc commands: /ping, /info, /stats also exist
❌ All other commands no longer exist.

🎉 Blacklist will now combine your personal blacklist and the server blacklist for your searches

Set up your personal blacklist by using /blacklist add tag or /blacklist add site in DMs
It will be applied to every search you make
Don't like cats? Blacklist cat and your searches will never show a cat ever again
Only use 1-2 sites? /blacklist add site site: all and then blacklist remove site the sites you only want to see

Search is now significantly better

🎉 Actual buttons that work better than reactions
🎉 Previous image button
🎉 Hidden images can be unhidden and don't break previous/next buttons
🎉 Only the searcher can use the buttons
🎉 You can now privately search (and decide to publically post any result you want, in case you want to show people something specific)

Config settings are different now

⚠ minScore → min_score (It still works the same)
❌ topicEnabled is now gone, use Server Settings > Integrations > BooruBot > Commands

Select /search, then set "All Channels" to "Deny". Add the channels you want people to search in.
If you have admin perms, you'll always see the slash command (Discord limitation). Other people won't and can't use it.
It should look like this

❌ nsfwServer is now gone. Just mark your channels as age-restricted. You need to for TOS reasons, it was originally because you needed to name every channel #nsfw- (like, 6 years ago)
❌ disableNextImage is now gone. There's always buttons.
🎉 allow_nsfw is new, and decides if the bot posts any NSFW posts anywhere (posts not rated safe)

By default it's True on servers (post NSFW in age-restricted channels), set it to False to never post NSFW
It's False in DMs, use /config set allow_nsfw allow:True in DMs to get BooruBot to send you NSFW posts

The database was migrated

You probably won't be affected
min_score, blacklisted tags/sites should all be migrated
If they aren't for whatever reason, contact me (Atlas#2564) on the support server
I will be keeping the old database for ~month (end of September) just in case, and then deleting it. Contact me before then if something is missing (or you want to keep your data)

And many other changes only I am excited about, so I'll leave the changelog at that.
If you have any bug reports/feedback/suggestions about this update, you can leave a GitHub issue or message on the support server
I hope you enjoy, now I have to update 3 other bots :(


Add intro.ts demo that demos intro tutorial, fix intro to actually converge when training.

This change is

Fixes #22

Review status: 0 of 3 files reviewed at latest revision, 4 unresolved discussions.

demos/intro/index.html, line 1 at r1 (raw file):

Introduction tutorial code

Add copyright header

demos/intro/intro.ts, line 39 at r1 (raw file):

    // Blocking call to actually read the values from average. Waits until the
    // GPU has finished executing the operations before returning values.
    console.log('mean squared difference: ' + average.get());  // average is a Scalar so we use .get()

wrap to 80 width

demos/intro/intro.ts, line 58 at r1 (raw file):

  const labelTensor = g.placeholder('label', labelShape);

  // Variables are containers that hold a value that can be updated from training.

wrap line

demos/intro/intro.ts, line 133 at r1 (raw file):

    console.log('value: ' + testOutput.get(0));
  });
}

should we dispose all inputs and labels here? or too much detail for this tutorial?

Comments from Reviewable


"Unable to read the message body." intermittently

version 4.0.110
I am using the Event Hub Listener to view events in an Event Hub.  When the events are initially consumed, I can usually click the row in the Events List grid, and the event text and custom properties are shown correctly.  However, clicking off this row and back, it gives the message "Unable to read the message body", plus the custom properties are missing.   Also, this message is sometimes shown even the first time the event is viewed.   I can restart the listener, and the events show correctly again, but this error comes back frequently.  Could it be because I am using JSON as the event body (not XML)?

I am seeing this issue as well. Is there a fix or work-around for it?

It may be due to timeouts. There is one timeout that by default is set to 1 second. @japicket and @davidwmartines please increase the Service Bus Explorer timeout settings and report back here whether that solves the this issue or not


[Removal]: Can I have a second chance?

Hotspot Name
Little Coconut Starfish
Hotspot b58 Address
112eh4MWHQrkaxbew4jhtTx1cXPncJcPcUfx9GvQ
Discord Handle
ev-Demon#3524
Hotspot Manufacturer
Bobcat
Removal Reason
Can I have a second chance?
Modifications
No
Extra forwarders
No
Extra antennas
No
Additional Information
I'm sorry that I've done this. I bought these bobcats back in August of last year and I got them just a few months ago. between august and when I got the hotspots the area got super crowded and profit come down a lot. I was barely making some HNT. I didn't know that there was a Denylist. Now I know. :(
I have all the hotspots in different locations, the only thing is that I placed them in a different location in the map to be able to get a TS of 1.
If I can get a second chance I will move the hotspots to the real place. I think that Helium is an excellent project and I would like to keep being part of it. Please consider my petition for denylist removal. Thank you.
Here is a pictures of the antenna at my house. I can send pictures of the other locations if you want.

Please please, accept my apology and remove me from the denylist.

Invalid HS address.
Correct address 112eh4MWHQrkaxbew4jhtTx1cXPncJcPcUfx9GvQzqT3KXg8SGyj

Thank you.

Hi, I wanted to follow up on this. Let me know. Thank you.


Reset the gauges on beginning of request.

This causes the prometheus client exporter to calculate the different between the current value and the previous value?

This is more for prometheus itself.  What happens here is the gauge will go to zero on reset for all labels and we will then adjust based on any stats we may have.  If we don't reset: let's say a guage on the previous request had a value of 4 and on the next request we have no stat we would want the stat to be zero, but the guage will just stay at 4 which tells prometheus that the current value is still 4.  There is another interaction that happens if on a previous request the gauge is 4 and then on the following we request we don't send a value at all for the gauge, prometheus itself will think the value is still 4.  So zeroing/resetting is needed for both situation.
TLDR Prometheus is weird.


WALLET_LOCKED error?

I'm getting this error from BWS in my copay client. What's causing this error?

Same here:-(
" Could not create transaction"
"WALLET_LOCKED: Wallet is locked"
What that means? I lost some money now because of the exchange rate. That is not accaptable!
What is the issue?

Same issue - looks like an urgent problem. All BWS users might be affected.

please follow https://github.com/bitpay/copay/issues/6930, we are investigation the issue


Sass changes

Changes to default Sass:

Move hardcoded values to variables
Deprecate viewport mixin via @warn
Update $min-mobile breakpoint to 420 from 480
Update $page-width to 1290px
Rename (and deprecate old) grid mixins to grid-col etc
Add heading() subheading and copy() mixinstemplates

Yeah, I had the scale thingie in mind. Has it been reviewed? Or should I just port it here and add to a page for easier review?

yeah @osk maybe add in from that PR & we can test here instead -- not sure it needs to be implemented for ueno.co anyway (or at least there is no urgent need)

It would be nice to add these to html {} in base.scss
-webkit-font-smoothing: antialiased; // stylelint-disable-line
-moz-osx-font-smoothing: grayscale; // stylelint-disable-line

I've updated according to comments.
@organdonor47 I'll add the scale experiment in a new branch after this one is merged.

On second thought, added a first version of scaling to responsive-font

@k80 @organdonor47
I removed the @warn from the mixins and set explicit $arbitrary-* variables instead it would always be changed, right? 😊

I'm merging this later today if there are no objections

i think there is probably a more sensible solution to this down the line, but this is as good as any other alternative I can think of!


Estonian translation

Skipped untranslatable strings and currently tooltips as well, but all visible strings should be translated.

real gaming

Thanks, that's great!


[CSL-1549] correct enqueue subset

Peers which are not routed in the queue's peers term now appear in the
'unknown' set and are appropriately classified and routed.

CI exceeded maximum time limit D:

Didn't even have time to build the dependencies stack --no-terminal --install-ghc build --only-dependencies --jobs=1. Could we bump up that --jobs option?

Anyway, force pushed to try the builds again and my fingers are crossed.

@dcoutts nice, thanks for that travis update. I was wondering if there was a good reason for using -j1.


Improve support for Schema Registry in DC/OS

Properly co-locate Schema Registry zookeeper info
in the namespace of the Confluent Kafka service
to which the zookeeper is deployed.   Original version did not do this ... so
multiple schema registry services against different Confluent clusters in the
same DC/OS environment could conflict with each other.
No change to resource.json necessary (Docker image remains the same).
** changes for new version **

diff --git a/HEAD:./5/config.json b/HEAD:./6/config.json
index 6763573..a5f43e3 100644
--- a/HEAD:./5/config.json
+++ b/HEAD:./6/config.json
@@ -38,9 +38,14 @@
           "description": "Deploy schema-registry only on nodes with this role.",
           "type": "string"
         },
-        "kafkastore-connection-url": {
-          "default": "master.mesos:2181/dcos-service-confluent-kafka",
-          "description": "Zookeeper Connect string for service cluster. Format is comma-separated list of :/",
+        "zookeeper-master": {
+          "default": "master.mesos:2181",
+          "description": "Zookeeper Connect string for service cluster. Format limited to single target: :",
+          "type": "string"
+        },
+        "kafkastore": {
+          "default": "dcos-service-confluent-kafka",
+          "description": "Name of the Kafka service hosting the storage for this Schema Registry edition.",
           "type": "string"
         }
       },

diff --git a/HEAD:./5/marathon.json.mustache b/HEAD:./6/marathon.json.mustache
index ff5bc78..f09ebbe 100644
--- a/HEAD:./5/marathon.json.mustache
+++ b/HEAD:./6/marathon.json.mustache
@@ -30,7 +30,8 @@
   } ],
   "env": {
     "SCHEMA_REGISTRY_HEAP_OPTS": "-Xmx{{registry.heap}}M",
-    "SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL": "{{registry.kafkastore-connection-url}}"
+    "SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL": "{{registry.zookeeper-master}}/{{registry.kafkastore}}",
+    "SCHEMA_REGISTRY_SCHEMA_REGISTRY_ZK_NAMESPACE": "{{registry.kafkastore}}/{{registry.name}}"
   },
   "healthChecks": [
     {

diff --git a/HEAD:./5/package.json b/HEAD:./6/package.json
index 7e970be..655a52f 100644
--- a/HEAD:./5/package.json
+++ b/HEAD:./6/package.json
@@ -1,7 +1,7 @@
 {
   "packagingVersion": "3.0",
   "name": "confluent-schema-registry",
-  "version": "0.9.8-3.2.0",
+  "version": "0.9.8-<IP_ADDRESS>",
   "scm": "https://github.com/confluentinc/schema-registry",
   "description": "Confluent Schema Registry service\n\n\tDocumentation: http://docs.confluent.io/3.2.0/schema-registry/docs/intro.html",
   "maintainer"<EMAIL_ADDRESS>

Did you want a version different from the current change :

-  "version": "0.9.8-3.2.0",
+  "version": "0.9.8-<IP_ADDRESS>",

I was going back and forth between <IP_ADDRESS> or the bumping the serviceVersion to 0.9.9

ah! Just checked that the baseTechVersion (Schema Registry's version) is 3.2.0
lets make it 0.9.9-3.2.0

Done.


iterateUntilM evauates before the loop ?

untilM evaluates the condition after the loop, keeping all results.
iterateUntilM should behave the same (judging from the name), except that it returns only the final result.
However, it seems to evaluate the condition before the loop:
iterateUntilM :: (Monad m) => (a -> Bool) -> (a -> m a) -> a -> m a
iterateUntilM p f v
    | p v       = return v
    | otherwise = f v >>= iterateUntilM p f

That confuses me. Is the naming to be unfortunate or the implementation to be incorrect?

I don't think that this is a bug. The current semantics is intuitively clear: We want to iterate starting with a, until some condition. If the condition is already true we do not have to iterate.
But the fact that iterateUntilM is infix operator is indeed a bug. It should not.


Documentation on custom loss function

I would like to have a custom loss function for my neural network. Would it be possible to have some documentation on how to do this? Thanks!

After a bit experimentation with the vae and mnist examples, this is actually rather easy to do. Thanks for the great library!


Clarification on documentation

In regards to this line:
To be backwards compatible with the current version of group writeback, when you enable group writeback, all existing Microsoft 365 groups are written back and created as distribution groups, by default.
Is this referring to the the group's writing back "as default" because their default value is null? Or is it stating that when the writeback V2 is turned on that it is going to force all 365 groups to a null value, even if they're currently true or false?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0ef5e848-ca7a-8924-8da2-73163a489e5b
Version Independent ID: ad118082-f4bb-624c-b541-942ba235f370
Content: Azure AD Connect: Group writeback - Microsoft Entra
Content Source: articles/active-directory/hybrid/how-to-connect-group-writeback-v2.md
Service: active-directory
Sub-service: hybrid
GitHub Login: @billmath
Microsoft Alias: billmath

@justinmontgomeryCF
Thanks for your feedback! We will investigate and update as appropriate.

Hi @justinmontgomeryCF
In this context, "written back" refers to the process of synchronizing data from Azure Active Directory (AAD) to your on-premises Active Directory (AD). By default, Microsoft 365 groups are not synchronized to AD, so when you enable group writeback, these groups will be "written back" to AD as distribution groups, which is the default group type in AD.
This means that the existing Microsoft 365 groups will be synchronized to AD as distribution groups, regardless of their current group type in Microsoft 365. This does not affect the value of any other properties of the groups, such as their default value or any other true/false value.
if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

Hello @AjayBathini-MSFT
OK, so it's just talking about the group type.
So it will writeback all existing unified groups as distribution when it's enabled. That will be true even if the OnPremiseGroupType is set to a different group type?

@justinmontgomeryCF
Yes, that's correct. When you enable group writeback, all existing Microsoft 365 groups will be written back and created as distribution groups in the on-premises Active Directory, regardless of their current group type. This means that if the OnPremisesGroupType of an existing group is set to a different value (such as Security or Mail-Enabled Security), it will be changed to Distribution when the writeback occurs.


Add Unit tests for server.rs

Part of #3

I would suggest we divide the unit test for server into three categories

saftey
leader election
log replication
We can start with asserting the state of server.

Hi, I would love to take this issue. Can you please assign the issue to me? I will come back with MR soon.

@amustaque97 any progress on this issue?

@vaibhawvipul I was busy for a while, will submit the MR before next week.

Hey @amustaque97 Thanks for you efforts. I see this is long pending, Would you mind if i take over this issue ?

Hey @amustaque97 Thanks for you efforts. I see this is long pending, Would you mind if i take over this issue ?

Thanks for reaching out! Yes, please you can take it over. Apologies, I'm a bit occupied with something else.


init zeroclipboard  throw flash-disabled error

This software uses flash, but flash is a security risk and shouldn't be used anymore! You could use the clipboard apis (https://www.w3.org/TR/clipboard-apis/)


profiling/benchmarking work

This PR is the result of my efforts on closing #32. It probably looks like a lot, but it's not, really. Here's a summary:
🔹 Shuffled some code around to strengthen the boundary between definitions and implementations
Previously there was a mix of definition and implementation between the hpp and cpp files, which is likely to become a problem if/when the project grows in size, so I moved some of the heavier implementation code from pytomlpp.hpp to more specific cpp files.
🔹 Added a benchmark script with some toml test data
It's basically a slightly more complex version of the benchmarking example you have in the README. I also added tomlkit to it for the fun of it. Output looks like this:
  pytomlpp.loads:   0.13249060000000057 s
      toml.loads:   1.1993466000000002 s
   tomlkit.parse:   7.730476 s

The test data was generated by one of the toml++ demo projects, toml_generator, and can be used to generate much, much larger toml files if you really want to stress-test :P
🔹 Hid all the profiling stuff behind a compile-time switch
As well as making it auto-report on script exit. This will mean you don't need to explicitly access anything to do with the profiling machinery from inside of python; all you need to do is set PYTOMLPP_PROFILING to 1 in pytomlpp.hpp and rebuild, then run any script using pytomlpp, and see a report on shutdown:
pytomlpp profiling summary:
-----------------------------
    loads.total:  counter =    1000, total_time_in_ns =    129455600, average_time_in_ns =  129455
  loads.convert:  counter =    1000, total_time_in_ns =     32181100, average_time_in_ns =   32181
    loads.parse:  counter =    1000, total_time_in_ns =     90427300, average_time_in_ns =   90427

You'll note I also added a scope timer for the toml::parse() section, and see that it accounts for most of the time spent in loads.
🔹 Simplified toml_array_to_py_list and toml_table_to_py_dict
They're now much less complex, using visitation instead of the previous if/else chain. They also previously had cases that were impossible to reach; a toml value can never have type node_type::none or anything not already handled by one of the other conversion paths, so those branches were dead code.
🔹 Updated toml++ to 2.1.0
I actually encountered an upstream bug while working on this that required an update, heh.
Overall the performance outcomes were negligible since it appears parsing accounts for most of the time spent, which was a bit disappointing, though considering the speed compared to tomlkit and toml I think you're OK in that department :P

This is GREAT! I will merge this now. Thanks!

No worries. I had fun working on this. Good to get out of C++-land every now and again.
You can probably close #32, methinks; Given that most time is spent parsing, the only way I see pytomlpp getting any faster is if I do upstream work to make toml::parse faster, but I already do perf passes on it quite regularly and am pretty much out of ideas (without moving to something nuts like a SIMD-based unicode decoder or something...)
Besides, you're 9x faster than toml and 59x faster than tomlkit, heh.

Oh, something I didn't add, though you may wish to consider:
add CI check to ensure nobody can accidentally commit #define PYTOMLPP_PROFILING 1


Remove time from date variables with no times in subset modal download files

In the download files made by the modal, the user can download dates. Usually (maybe always), these dates have no time associated, but the date appears as 2017-03-14T00:00:00 (ex LLINEUP "Observation date" variable). When there are no times entered, we want to strip out "T00:00:00" from the values.

Yup, times have been stripped. QA is complete.


Add header elevation option

Possible solution for #13
When another header is displacing a current one, the elevation difference looks a bit off. Maybe elevation should be reset if a header is being displaced? That doesn't look quite right either.

Now elevation is animated. Elevation is removed if a header is being pushed by another header.
I think the overall effect is pretty good.


Translation Event not produced for some configurations which result in Gateway returning 500

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Some configurations can cause KIC to not produce Events (with reason KongConfigurationApplyFailed) which should be produced.
Expected Behavior
In case of an incorrect configuration, an Event with KongConfigurationApplyFailed is produced indicating what has failed.
Steps To Reproduce

Apply the following configuration
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx-sample-1
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-sample-1
  template:
    metadata:
      labels:
        app: nginx-sample-1
    spec:
      containers:
      - image: nginx
        name: nginx-sample-1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
          name: http
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    konghq.com/methods: GET
    kubernetes.io/ingress.class: kong
  name: nginx-sample-1
  namespace: default
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: nginx-sample-1
            port:
              number: 80
        path: /sample/1
        pathType: ImplementationSpecific
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    ingress.kubernetes.io/service-upstream: 'true'
    konghq.com/plugins: nginx-sample-1 #,nginx-sample-1-rate # removing this breaks Kong but it does not validate on webhook
  name: nginx-sample-1
  namespace: default
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx-sample-1
---
apiVersion: configuration.konghq.com/v1
config:
  allow:
  - nginx-sample-1
  - app
  hide_groups_header: false
kind: KongPlugin
metadata:
  name: nginx-sample-1
  namespace: default
plugin: acl
---
apiVersion: configuration.konghq.com/v1
config:
  allow:
  - nginx-sample-2
  - app
  hide_groups_header: false
kind: KongPlugin
metadata:
  name: nginx-sample-2
  namespace: default
plugin: acl
---
apiVersion: configuration.konghq.com/v1
config:
  fault_tolerant: true
  hide_client_headers: false
  limit_by: consumer
  policy: local
  second: 2000
kind: KongPlugin
metadata:
  name: nginx-sample-1-rate
  namespace: default
plugin: rate-limiting
---
apiVersion: v1
data:
  kongCredType: YmFzaWMtYXV0aA==
  password: NmVmNzI4ZGUtYmE2OC00ZTU5LWFjYjktNmU1MDJjMjhhZTBi
  username: Nzc0Zjg0NDYtNjQyNy00M2Y5LTk5NjItY2U3YWI4MDk3ZmU0
kind: Secret
metadata:
  name: app-cred
  namespace: default
---
apiVersion: v1
data:
  group: YXBw
  kongCredType: YWNs
kind: Secret
metadata:
  name: app-acl
  namespace: default
---
apiVersion: configuration.konghq.com/v1
config:
  fault_tolerant: true
  hide_client_headers: false
  limit_by: consumer
  policy: local
  second: 2000
kind: KongPlugin
metadata:
  name: nginx-sample-2-rate
  namespace: default
plugin: rate-limiting
---
apiVersion: configuration.konghq.com/v1
credentials:
- app-acl
- app-cred
kind: KongConsumer
metadata:
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/plugins: nginx-sample-1-rate,nginx-sample-2-rate # This is necessary for nginx-sample-1 Service to break Kong's config
  name: app
  namespace: default
username: 774f8446-6427-43f9-9962-ce7ab8097fe4

Observe in KIC logs an error about pushing configuration
 time="2023-04-21T16:48:52Z" level=error msg="could not update kong admin" error="performing update for https://<IP_ADDRESS>:8444 failed: failed posting new config to /config: got status code 500\nperforming update for https://<IP_ADDRESS>:8444 failed: failed posting new config to /config: got status code 500" subsystem=dataplane-synchronizer

Observe errors logs in Gateway about internal server error
2023/04/21 16:49:43 [error] 1204#0: *299 [lua] api_helpers.lua:516: handle_error(): /usr/local/share/lua/5.1/lapis/application.lua:424: /usr/local/share/lua/5.1/kong/db/errors.lua:790: attempt to index local 'entity' (a string value)
stack traceback:
      /usr/local/share/lua/5.1/kong/db/errors.lua: in function 'add_entity_errors'
      /usr/local/share/lua/5.1/kong/db/errors.lua:797: in function 'add_entity_errors'
      /usr/local/share/lua/5.1/kong/db/errors.lua:853: in function 'flatten_errors'
      /usr/local/share/lua/5.1/kong/db/errors.lua:970: in function 'declarative_config_flattened'
      /usr/local/share/lua/5.1/kong/api/routes/config.lua:136: in function 'fn'
      /usr/local/share/lua/5.1/kong/api/api_helpers.lua:287: in function </usr/local/share/lua/5.1/kong/api/api_helpers.lua:270>
stack traceback:
      [C]: in function 'error'
      /usr/local/share/lua/5.1/lapis/application.lua:424: in function 'handler'
      /usr/local/share/lua/5.1/lapis/application.lua:146: in function 'resolve'
      /usr/local/share/lua/5.1/lapis/application.lua:183: in function </usr/local/share/lua/5.1/lapis/application.lua:181>
      [C]: in function 'xpcall'
      /usr/local/share/lua/5.1/lapis/application.lua:189: in function 'dispatch'
      /usr/local/share/lua/5.1/lapis/nginx.lua:231: in function 'serve'
      /usr/local/share/lua/5.1/kong/init.lua:1572: in function 'admin_content'
      content_by_lua(nginx-kong.conf:320):2: in main chunk, client: <IP_ADDRESS>, server: kong_admin, request: "POST /config?check_hash=1&flatten_errors=1 HTTP/2.0", host: "<IP_ADDRESS>:8444"
<IP_ADDRESS> - - [21/Apr/2023:16:49:43 +0000] "POST /config?check_hash=1&flatten_errors=1 HTTP/2.0" 500 42 "-" "Go-http-client/2.0"

Kong Ingress Controller version
2.9.3

Kubernetes version
No response
Anything else?
No response

I'm not sure we could produce a useful Event here beyond "congrats, you found a bug! ask us about it!". The gateway error reporting itself failing produce a normal error report naturally discards information about what caused it, since if it was able to produce that information, it would not have failed.

For UX consistency we could produce a single "something went horribly wrong, call us, include the proxy container logs" Event in our own namespace that'd show up in kubectl searches.

@rainest How can we address this issue? This is impacting Westpac and delaying the deployment to the production environment. This puts the KIC in the reload loop.

@manibh-kong While we can improve error handling on our side of HTTP 500s I've created this issue for gateway w.r.t flatten_errors which we use for sending configs to gateways 3.2+: https://github.com/Kong/kong/issues/10767

One thing to keep in mind for this is that there's no Object to attach the event to when KIC receives a 500 from Gateway.
A typical 500 is "stuff went horribly wrong for unknown reason". Hence we can't use https://pkg.go.dev/k8s.io/client-go/tools/record#EventRecorder.Event unless we wan't e.g. to attach said Event to e.g. KIC's Pod.

For UX consistency we could produce a single "something went horribly wrong, call us, include the proxy container logs" Event in our own namespace that'd show up in kubectl searches.

I think this would be useful. Point them to GitHub to raise an issue with reproduction steps and then we can escalate to Gateway as needed

The simplest configuration to re-produce this bug is to create two different KongPlugin CRDs using the different plugins, and configure them in the annotation konghq.com/plugins of the same KongConsumer:

Handled in https://github.com/Kong/kubernetes-ingress-controller/issues/4003


Undefined, Undefined, Undefined, Undefined, Undefined, Undefined,

I think this happens when provider returns matches with prefix not contained in word, e.g.

I'll try to pinpoint the location. Most likely, it's highlighting issue.

Sorry about this, it's out in 2.3.1.


Converting a Dictionary to JSON

Is there a easy way to convert a dictionary (i.e of type [String : String], [String, AnyObject],...) without writing all this boiler plate?
var tempDict: [String : JSON]
dict.forEach { k, v in
    if let v = v as? String {
        tempDict[k] = JSON.String(v)
    } else if let v = v as? Int {
        tempDict[k] = JSON.Int(v)
    } else if let v = v as? Bool {
        tempDict[k] = JSON.Bool(v)
    }
}
let json = JSON.Dictionary(tempDict)

If not should we add a new initializer?

I ran into this too. Going up a level, you can get that dictionary out but keep it a JSON value:
try JSON(yourJSON.dictionary(keyForTempDictValue))
Another option is to just guard that your value is a dictionary, but not unpack it, with something like:
guard case .Dictionary = jsonvalueThatContainsTempDict
There are other approaches, but I'd have to know a bit more about what problem you're trying to solve. In my case, we are using JSONAPI, so I needed the attributes dictionary out and did not like the repetition of indexing into attributes as part of my keypath for each property. I ended up going with the first solution, of re-wrapping the dictionary back into JSON.

I'm not sure I understood what you said. Anyway the use case is that I have a dictionary and I want to create a JSON value. An easy example is: I have a dictionary in Core Data and I want to create a JSON value out of it.

Sorry, I misunderstood your question.

I think what you're asking for is pretty similar to how we interface with NSJSONSerialization: https://github.com/bignerdranch/Freddy/blob/dc98e18c84418cc984e283b1836dcb1bd5324d68/Sources/JSONParsing.swift#L57-L77
We've had a few discussions about making an interface like that public. The problem is that it's really, really slow for large objects, and we don't really want to encourage people to use it. We addressed the NSJSONSerialization case by making JSONParser the default parser. Could you talk more about your use case? In particular, what do you mean by "I have a dictionary in Core Data"?

So it's a very specific enterprise app, that user can customize with their own code written in JavaScript. One of the feature is that they can store objects that have a fixed part and a part that could be anything and it's stored in a dictionary. So the NSManagedObject looks something like this:
    @NSManaged private(set) var property1: String
    @NSManaged private(set) var property2: String
    @NSManaged private(set) var otherParameters: [String : AnyObject]?

now, again from JavaScript, they can search and they get back a json array containing all the objects matching the search. And that's where, from the NSManagedObject subclass, I create a JSON class (converting the Dictionary<String, AnyObject> to JSON) and then I need to create a string representing the JSON to pass to the webview evaluateJavaScript method.

Sorry, more questions. Is otherParameters a Transformable in your Core Data model? How does data get into otherParameters? Would it make sense to use a class that wraps a Freddy.JSON instead of dictionary? Finally, do you interact with otherParameters from Swift, or would NSJSONSerialization be a better solution to this problem?

Yes otherParameters is a Transformable and it gets its values from the JavaScript passing JSON to the Swift counterpart. There is some interaction from Swift with this otherParameters driven by the JavaScript code. This otherParameters also is passed as a dictionary to another swift framework.
Since this dictionary is generally very short, I prefer to have the boilerplate I pointed out above, then use for instance NSJSONSerialization. It makes the rest of the code cleaner.

Hi @lucatorella. Are you still looking for assistance on this issue, or can we close it?

Hi, I already have a working solution which is the one from my first comment. It just seems a bit like boilerplate.


[QUERY] Clean Up Session State with ServiceBusSessionProcessor

Library name and version
Azure.Messaging.ServiceBus 7.17.1
Query/Question
How are users intended to clean up session state when using a ServiceBusSessionProcessor?
When there are no messages in a session ServiceBusSessionProcessor simply stop raising events. The session state will persist indefinitely. There's no chance to check the state for an expiration and clean it up.
On the other hand ServiceBusSessionReceiver still accepts the session even if there are no active messages, affording an opportunity to clean up the session state.
Environment
No response

Thank you for your feedback.  Tagging and routing to the team member best able to assist.  Please expect and forgive slow responses during the US holiday season.

Hi @ben-page, you can subscribe to the SessionClosingAsync event so that you can execute code when the processor is done with a session. Note that this doesn't mean there will never be more messages for that session (this is entirely dependent on your application), but it does mean that the processor timed out based on the SessionIdleTimeout setting without receiving any more messages for that session.

/unresolve

@JoshLove-msft thank you for a the prompt response, but that doesn't quite address my question.
I'll elaborate. Cleaning up session state when the processor closes it isn't particularly useful. If the session state doesn't need to outlive the current receiver's lock on the session, why even use session state? You could just hold the state in the application.
I think the most common use case for session state is when you have some kind of pattern, such as a saga, where related messages will arrive at somewhat random intervals. But if something were to fail and the final message was never sent, then the session state would persist indefinitely.
At least that's the case for ServiceBusSessionProcessor, which appears to have no way to access a session that does not have active messages. Alternatively if you are using a ServiceBusSessionReceiver, you would still end up with a lock on the session though there are no messages. So if your session state stored an expiration, the application would have a chance to clean it up.

I think the most common use case for session state is when you have some kind of pattern, such as a saga, where related messages will arrive at somewhat random intervals. But if something were to fail and the final message was never sent, then the session state would persist indefinitely.

If this is the pattern you are using, could you clean up the session state when the final message arrives?  Or is the cleanup based on some expiration time?

At least that's the case for ServiceBusSessionProcessor, which appears to have no way to access a session that does not have active messages. Alternatively if you are using a ServiceBusSessionReceiver, you would still end up with a lock on the session though there are no messages. So if your session state stored an expiration, the application would have a chance to clean it up.

You can still specify specific SessionIds to process, in which case those sessions will be opened even if there are no messages available.


unstable channels have not been updated in over 10 days

Issue description
Both the nixos-unstable and nixpkgs-unstable have not been updated in over 10 days: https://howoldis.herokuapp.com/
However, there were at least some clean evaluations during that time. Any idea why the channels are stuck?

There really were no evaluations that succeeded – see the label at the top of the linked page:

This is an aggregate job: its success or failure is determined entirely by the result of building its constituent jobs. [emphasis mine] The table below shows the status of each constituent job for the 15 most recent builds of the aggregate.

Yeah, nixos-unstable seem to have been broken for one reason or another a while (though there's this eval https://hydra.nixos.org/build/86233229#tabs-constituents where I suspect the failing job was restarted, but the success not picked up).
The nixpkgs-unstable unstable job have been green a few times recently though: https://hydra.nixos.org/job/nixpkgs/trunk/unstable#tabs-status (as I understand it the nixpkgs-unstable channel should advance when this job passes, though I might be mistaken here).

python37 seems to have been broken by a patch not applying in https://github.com/NixOS/nixpkgs/commit/552d8a70d4c47a6e52a8e0b26e3a2e33d5b94ca6

About python3 https://github.com/NixOS/nixpkgs/issues/52821

I expect the nixpkgs-unstable channel to advance after 2534dddaa96af8faa442ba7ac360966c7990e773 is tested.

Some failures on nixos-unstable are more mysterious: https://hydra.nixos.org/build/86233229 ran all tests successfully (all green) but failed a depend installer test (https://hydra.nixos.org/build/86233229/nixlog/34). However, looking at the log of the test it is unclear why the test even failed. Every test within succeeded with exit status 0. Any ideas what goes wrong here?

nixpkgs-unstable has had some successes of the job since the last update, so I gather it's again some "invisible" error happening in the script that updates channels.  @edolstra: can you post the error somewhere?  (not sure who else can see these)

nixos-unstable got updated now.

Can we make nixos-unstable be dependent on pandoc? It is really unfortunate that it has advance with +10000 broken packages, all due to the pandoc failure.

Maybe we'd better add a threshold on percentage o failures.

All are up to date now.  No idea what was blocking the nixpkgs channels.

It looks some of the installer tests are failing again (?).


Collaborators

Dear

@B-Stefan
@EugeneHlushko
@roopemerikukka
@eek
@Andarist
@neeharv
@rmariuzzo
@timtyrrell
@zakness
@jedwards1211
@bramschulting
@gaearon
@danistefanovic
@neurosnap
@julienbachmann

thank you for your past contribution to this project!  If you would be interested to become a collaborator, I would love to collaborate with you :) Just put your name down and we can talk plans.
Next week I'll kick off a discussion around next steps, including:

improving demos & documentation
rewrite for version 4, and important considerations for upgrade path
your idea!

Thanks again for your help

@joshwnj I'm always too busy but I do have some ideas about what I'd like the version 4 API (or maybe just a minor release) look like, so I'll try to contribute, and count me in the discussion at least.  I'd also be happy to set up code coverage for the tests.
Also I was thinking a great demo would be one of those snazzy staggered animations that start when something first becomes visible.  I could whip one up at some point using react-motion.

@jedwards1211 great ideas, thanks.

Hi @joshwnj ,
If i found time count on me, we are getting tons of work lately in the company but who knows. I still think this repo is very useful to the community. 👍

Hi @joshwnj, I'm always with an eye around, especially since I'm using react-visibility-sensor in quite a big project, and would definitely love to see faster response times, I'm always using it from a fork which is annoying. haha xD

I am not currently using the project (I just noticed that bug I fixed when I was reviewing it for possible use), not interested but good luck 😁

@eek I'll be very glad to have your help - thanks! :)
@falcon1kr please consider yourself added to this invitation as well :)

hi @joshwnj , aint sure how much involved I can be, but I might try to do some work to the extents of my possibilities

@joshwnj Hi josh, I was looking into adding code coverage, but I've never used browserify before.  (Is there any particular reason you use it instead of webpack, or are you just more familiar with it?)
I have a react-library-skeleton project that supports code coverage and is ready to drop code into, so I could try putting your source/test code into it.  However, it has some major differences -- it uses babel, webpack instead of browserify, and jsdom instead of karma.  I'm not sure if jsdom completely simulates the scrolling and visibility needed for the tests, but I could find out.  It is faster than karma so that's a plus, but I can understand if you're nervous about whether it behaves the same way as real browsers.
Would you like me to experiment with putting your code into my skeleton?  I could also easily convert your code to ES2015 in that case.  If not I could look into using karma-coverage instead.

Hey @jedwards1211 sounds cool. Would be very interesting to try it out, but as it's quite a big shift away from the current codebase let's consider it a fork rather than a PR at this stage.

Is there any particular reason you use [browserify] instead of webpack, or are you just more familiar with it?

bit of both :) I think for this case at least, a lighter-weight option is better.

I'm not sure if jsdom completely simulates the scrolling and visibility

yeah I would question that too. I originally started out with jsdom tests for this but there were a few weird edge cases that made me switch to testing in a real browser.

I could also easily convert your code to ES2015

Are there any parts of ES2015 that you think would add significant value to this project?  I'm happy with it in general, I'd just want to make sure it was worth the extra tech overhead before going down that path here.

Are there any parts of ES2015 that you think would add significant value to this project? I'm happy with it in general, I'd just want to make sure it was worth the extra tech overhead before going down that path here.

For once React.createClass is not staying with us for much longer, from what I know.

@Andarist do you have any links on that topic? I've heard similar rumours but haven't yet seen anything in writing :)

I'd be happy to help modernize the code base.
In terms of deprecating createClass, here's the link to the PR that removes
it - https://github.com/facebook/react/pull/9232
We could run a codemod to speed up the process, rather than do it by hand.
On Thu 6 Apr, 2017, 4:34 AM Josh Johnston<EMAIL_ADDRESS>wrote:

@Andarist https://github.com/Andarist do you have any links on that
topic? I've heard similar rumours but haven't yet seen anything in writing
:)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/joshwnj/react-visibility-sensor/issues/72#issuecomment-292022097,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABsS4ciC3GyTOWAreB7HB2rcVyaunwx7ks5rtB3ugaJpZM4MyNtO
.

Thanks for the link @neeharv - I'm AFK for the next couple of days but will read when I get a chance :)

You can count me in! Currently I have very limited time for additional work but I truly hope that it will change soon :P

@joshwnj thanks for the invite. Count me in as well, although, as others, I may only be able to invest limited amount of time.

Thanks @roopemerikukka & @falcon1kr

Since a few folks (including me!) have chimed in about helping with modernising the codebase, I've created an umbrella issue for discussion about that - https://github.com/joshwnj/react-visibility-sensor/issues/75


Use file.fullPath for BucketObject names, provide key for relativePath

Resolves #8

@sarink thanks again for reporting this, I've charted a path to a migration and described the error in detail here:

#10

In the mean time, I plan to release a new version with a fix that shouldn't cause any unwanted resource deletions or misbehavior. This is the pull request to implement that:

#11


Earliest Possible Date for Completed Conversion Tool

What is the earliest date that the conversion tool would be ready for 2024? Last year it was early December just want to gauge where we are.
Thank you.

Hi @mmveradigm, are you referring to the user testing or submission open date?  User testing - we are targeting by end of October.

Hi @sivaksb , I believe user testing .... just wanting to know when the update for 2024 would be available so that we can pull it and update our tool to confirm if QRDA files will convert to qpp.json properly.

@mmveradigm The PY24 year changes are in develop branch.  Just note that, the entity IDs are not updated yet in this file - https://github.com/CMSgov/qpp-conversion-tool/blob/develop/converter/src/main/resources/pcf_apm_entity_ids.json.  We are expecting this data will be updated after October 28.

Hello, when will the converter tool update be available for download? Last year we did not get notification until the first week of December.  This is the notification we got last year. Will we get something similar?

Hello,
@sivaksb , are the updates to the tool now complete?

@mmveradigm Yes it is complete and ready for preliminary testing.


⚠ Bofrost.de has degraded performance

In b990c9a, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1824 ms

Resolved: Bofrost.de performance has improved in fdd6cd1.


ResponseHandlingInputStream reads the rest of the stream on close

When close method is called ResponseHandlingInputStream loops until it reads the whole stream which can take too long time. Is there a way to disable this or a workaround?

This is needed in order to show the full response. Additionally there are issues when you dont consume the entire response in certain http engines.

Hmm, so what would you recommend to do, not to wait until >100MB file downloaded when download canceled?  This obviously affects how the app works. Do you think a proper solution implemented (I might develop and send a pull request) or should just fork the project do some hack?

+1. Reading the full response seems fine for loading a web page, but if you're making requests into a huge media file this is pretty problematic.
I think http engines typically consume the full response if the remaining length is below some threshold, so that the socket can be re-used. If the remaining length is greater than the threshold they'll tear down the socket.
If an engine does require that the whole of the response is consumed, that should really be implemented within the engine itself (i.e. in close of their own InputStream implementation). It doesn't make much sense to rely on every caller correctly draining the InputStream when reading. Can you point to a concrete example of an http engine that fails to do this and does require the caller to consume the response in full? A source code link would be cool; if it's an open source engine ;).

@ojw28  you are correct, the issues I was speaking about are directly around socket reuse. The fact that we install this is because internally we noticed engineers not reading all of the response due to some business logic that basically meant they were not going to need everything - even if it was already on the client. By not fully consuming the stream the socket got torn down which caused more issues than just reading it all. This had the additional benefit that, in stetho, we would represent the full response - which prevents developers from being very confused as to why, for example, the json is not complete.
@erdemguven  Would you want to disable on a single request/response or in stetho completely? Disabling completely would be a bit more simple as we could add an option to the constructor of the StethoInterceptor.
Are you using OkHttp? From the documentation it sounds like Call.cancel() will cause an IOException to occur on the read.

Having a parameter to disable it completely would be great. I tried Call.cancel() but somehow it breaks all connections after a couple of calls. Also I couldn't find a way to inject that call dynamically as I don't want to do that on the release build.

@erdemguven the preferred way to do this is to use something like Call.cancel in okhttp or implement your own cancelation mechanism on top of URLConnection which itself doesn't properly support cancelation.


Error downloading gwen-web-2.48.0

Hi,
Just launched gwen and I have the following issue :
[gwen-gpm] Downloading gwen-web-2.48.0 from https://github.com/gwen-interpreter/gwen-web/releases/download/v2.48.0/gwen-web-2.48.0.zip
java.io.FileNotFoundException: https://github.com/gwen-interpreter/gwen-web/releases/download/v2.48.0/gwen-web-2.48.0.zip

Maybe a typo with the name of the file identified as gwen-web-2.48.8.zip on the release page :wink:
Thanks.

Hi,
We've recently started hosting the binaries in maven repo's only and not here on GitHub as we used to. To fix your issue, perform the following

Download the latest gwen-workspace.zip from here: https://github.com/gwen-interpreter/gwen-web/releases/latest/download/gwen-workspace.zip
Extract it to a temporary location
Replace the gwen-gpm.jar in your current workspace with the one in this updated workspace

We've fixed the typo in the 2.48.0 release link too. Thank you for reporting it.

Thanks for the fix ! Everything is OK now.


[WIP] [api] Enhancement to return id as is and add href's seperately.

removed id normalization
added href's to json's via hash normalization
bumped up to ManageIQ v2.0.0-pre versioning

https://trello.com/c/Drza2h1q
Marked as WIP: More testing to do, as well as writing specs.

Code looks good to me, aside from my minor comments!


Add Redux for State Management

We implement Redux for state management within the service UI in order to facilitate the UIs working more closely together.

[ ] Migrate appliance info setting to utilize Redux

Change ApplianceInfo.set(response) to not have to run between page changes. Only set that when someone logs in and use redux to set and retrieve the appliance info.

[ ] Migrate functionality to set language to utilize Redux

Update how we persist selected language to use Redux. This needs to change in such a way that removes this notion of having to set language on page reload. Language factory should support retrieving selected language from redux when requested. This means deprecating Language.onReload(response)

[ ] Migrate functionality for LoadUser to use Redux

Move all functionality related to a user into its own service and remove it from Session Factory
Set this use redux to store user data between page reloads

Closing this, tracking in: https://bugzilla.redhat.com/show_bug.cgi?id=1502778


Feature 13412 - Support for sbc-staff user

Issue #: /bcgov/entity/issues/13412
Description of changes:
Support for sbc-staff user
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the bcrs-entities-create-ui license (Apache 2.0).

I don't think this is necessary - there is no existing SBC-Staff keycloak role (STAFF exists but not SBC-STAFF). They're a single org - it makes it quite a bit easier when they can manage the org themselves
https://github.com/bcgov/sbc-auth/pull/2089

New PR: https://github.com/bcgov/business-create-ui/pull/454