Eslint default and pr template

Description
Updating code base to use current ESLint & Prettier rules. This meant a lot of code needed to change, which lint:fix and prettier:write commands did most of the work. I fixed all the errors which couldn't be fixed automatically.
Also added PR template.
Changes

Updated esLint config with our standard rules
Updated Prettier with our standard rules
Updated package.json with packages needed for our Eslint standard rules.
Fix commands fixed most issues with code.
Afterwards, I manually combed through the files and fixed any lint errors that were left. mostly undefined and defined and not used.
Added a PR template for future PRs.

Side note: Build will fail until PR https://github.com/CruGlobal/campus-contacts-web/pull/870 is deployed.

From review it looks fine, but was waiting for #870 to put one here officially.

What needs to happen for the Amplify preview to work?

They are working as I can see the preview in AWS. Yesterday I had to reconnect Amplify to this Repo, so that might have been the issue.

Hmm, I can't get to that one. The one I see in the console failed:

2023-10-12T17:48:58.729Z [WARNING]: error<EMAIL_ADDRESS>The engine "node" is incompatible with this module. Expected version ">=16.0.0". Got "14.21.3"
2023-10-12T17:48:58.735Z [WARNING]: error Found incompatible module.
2023-10-12T17:48:58.736Z [INFO]: info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
2023-10-12T17:48:58.752Z [ERROR]: !!! Build failed

I just updated the build image which is now on node 16. Now it should work.
I'm also testing the staging site with the node 16 build image.


Remove MVCC import

Codecov Report

Merging #1346 (1d3a6c7) into master (51ca36f) will decrease coverage by 0.24%.
The diff coverage is 68.75%.

@@            Coverage Diff             @@
##           master    #1346      +/-   ##
==========================================
- Coverage   47.16%   46.91%   -0.25%
==========================================
  Files         202      201       -1
  Lines       15831    15700     -131
==========================================
- Hits         7466     7366     -100
+ Misses       7386     7362      -24
+ Partials      979      972       -7

Impacted Files
Coverage Δ

onboard/inventory.go
72.54% <ø> (-12.07%)
:arrow_down:

onboard/import.go
62.06% <60.00%> (-6.05%)
:arrow_down:

onboard/catalog_actions.go
67.39% <69.76%> (-1.50%)
:arrow_down:

catalog/mvcc/cataloger_create_repository.go
62.96% <0.00%> (-3.71%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51ca36f...1d3a6c7. Read the comment docs.

Codecov Report

Merging #1346 (1d3a6c7) into master (51ca36f) will decrease coverage by 0.24%.
The diff coverage is 68.75%.

@@            Coverage Diff             @@
##           master    #1346      +/-   ##
==========================================
- Coverage   47.16%   46.91%   -0.25%
==========================================
  Files         202      201       -1
  Lines       15831    15700     -131
==========================================
- Hits         7466     7366     -100
+ Misses       7386     7362      -24
+ Partials      979      972       -7

Impacted Files
Coverage Δ

onboard/inventory.go
72.54% <ø> (-12.07%)
:arrow_down:

onboard/import.go
62.06% <60.00%> (-6.05%)
:arrow_down:

onboard/catalog_actions.go
67.39% <69.76%> (-1.50%)
:arrow_down:

catalog/mvcc/cataloger_create_repository.go
62.96% <0.00%> (-3.71%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51ca36f...1d3a6c7. Read the comment docs.


Communication with Beckhoff virtual machine from another virtual machine

Hello guys :)
I I try to connect and read data from plc which is running in TwinCat3 BSD machine. I want to connect from another virtual machine  (Linux Ubuntu) using python and pyads module. Here is all stuff and information:
TwinCat3 BSD machine:

Search adapters information from TwinCat3 on Windows (my local machine):

Connection information from TwinCat3 on Windows (my local machine):

Code in python on virtual machine with Linux Ubuntu:
import pyads

if __name__ == "__main__":
    remote_plc_ip = '<IP_ADDRESS>'
    remote_plc_net_id = '<IP_ADDRESS>.1.1'

    SENDER_AMS = '<IP_ADDRESS>.1.1'
    HOSTNAME = '<IP_ADDRESS>'
    PLC_USERNAME = 'Administrator'
    PLC_PASSWORD = '1'
    PLC_IP = '<IP_ADDRESS>'

    pyads.open_port()
    pyads.set_local_address(SENDER_AMS)
    pyads.add_route_to_plc(SENDER_AMS, HOSTNAME, PLC_IP, PLC_USERNAME, PLC_PASSWORD)

    plc = pyads.Connection(remote_plc_net_id, pyads.PORT_TC3PLC1, remote_plc_ip)
    plc.open()

    if plc.is_open:
        print('Plc is open!')

    try:
        plc.read_state()
        print('Connection succeeded')
    except Exception:
        print('Connection failed')

    plc.close()

And now when i try run python code i get:
Error: Connect TCP socket failed with: 111
Traceback (most recent call last):
  File "/home/carolinii/Projects/energy_institute_scada/scada-server-side/device_servers/ads/test_ads_connection.py", line 18, in <module>
    plc.open()
  File "/home/carolinii/.local/lib/python3.10/site-packages/pyads/connection.py", line 198, in open
    adsAddRoute(self._adr.netIdStruct(), self.ip_address)
  File "/home/carolinii/.local/lib/python3.10/site-packages/pyads/pyads_ex.py", line 167, in wrapper
    return fn(*args, **kwargs)
  File "/home/carolinii/.local/lib/python3.10/site-packages/pyads/pyads_ex.py", line 189, in adsAddRoute
    raise ADSError(error_code)
pyads.pyads_ex.ADSError: ADSError: target port not found   ADS Server not started (6).

Anyone have idea what is wrong?

Disable firewall in TwinCAT BSD:

doas service pf stop
Note: The firewall is automatically re-enabled after each restart. This behavior is ensured by the entry pf_enable="YES" in the rc.conf file.

Check and adjust ADS routes manually in TwinCAT BSD using StaticRoutes.xml

Try this:
 plc = pyads.Connection(remote_plc_net_id, pyads.PORT_TC3PLC1)

Hope this will help!

@X-Carolinii-X if this fixed your issue, please can you close this issue to help us manage the issues log, thank you


Error upon nvim launch

Error detected while processing function codeium#server#Start[36]..53_ActuallyStart:
line   16:
E903: Process failed to start: no such file or directory: "/home/user/.codeium/bin/662505c9b23342478d971f66a530cd102ae35df7/language_server_linux_x64"

Re-download server from https://github.com/Exafunction/codeium/releases/

I got the same error and re-downloading doesn't seem to help.


Add a list of all coding challenges with a link to his folder

Hi, I've updated the readme.md to have a link between each folder and the corresponding coding challenge so it's easier to track which ones need to be ported from one language to the other. it's still pending to add a link for each coding challenge to each video on YouTube ;)

This is amazing, thank you!


Particle-falling. -effect by vanilla js

Project title
Particles falling effect by vanilla js
Project Description
Any base64 image can be converted into small small particle and it will create particle falling effect on the image.
Stack:
And in order to tick the check box just but x inside them for example - [x] like this. Please delete options that are not relevant.

[x] Html
[x] CSS
[x] JavaScript

/assign


Get image is broken in 0.9.0

Hi,
I'm trying to retrieve my uploaded images through the API, but it appears to be completely broken.
I'm POSTing the following JSON against sitename/api/cockpit/image
{
src: "cockpit/storage/uploads/images-4.png",
mode: "thumbnail",
width: 500,
height: 500,
quality: 80
}
and it always returns:
{
"error": "Target width or height parameter is missing"
}
I've tried different values for "src", I also tried the syntax from the documentation:
{
src: "cockpit/storage/uploads/images-4.png",
m: "thumbnail",
w: 500,
h: 500,
q: 80
}
But this doesn't work either

I can't verify the issue 🤔
Can you try the api directly via GET:
/api/cockpit/image?src=cockpit/storage/uploads/images-4.png&w=50&token={token}

Weirdly enough that works. But putting the exact same parameters in a json post doesn't 🤔

can you post the exact code snippet?

This is the raw request:
POST /api/cockpit/image HTTP/1.1
Host: vbox2:4061
Content-Type: application/json
Authorization: Bearer 0cae1123088d40b51540904a86a716
User-Agent: PostmanRuntime/7.11.0
Accept: /
Cache-Control: no-cache
Postman-Token: 24091b8c-5768-4a5a-a721-591635c3d4c0,ae7c29db-cf5e-43a5-bfd9-36c768908e4b
Host: vbox2:4061
cookie: mysession=5fcbaaa8d5a744a01715b023bb2c6a0f
accept-encoding: gzip, deflate
content-length: 52
Connection: keep-alive
cache-control: no-cache
{
"src": "storage/uploads/images-4.png",
"w": 50
}

🤷♂
curl -X POST \
  http://cockpit.local/api/cockpit/image \
  -H 'Authorization: Bearer b7e6aae549dedc4c7ef026dd6a9a7e' \
  -H 'Content-Type: application/json' \
  -H 'Postman-Token: bb9d0950-1ffa-4cf3-8850-0967571ff8b2' \
  -H 'cache-control: no-cache' \
  -d '{
"src": "storage/uploads/960.jpg",
"w": 50
}'

works for me.

I've just destroyed and rebuilt the docker image it was running in, and now it works...


Provide a way to remove array from Polymer.Collection (memory leak?)

From @danimoh on August 1, 2016 9:24
Hello.
I can register interest in the tracking of keys of array items by Polymer.Collection.get(myArray).
However, it can happen that you don't need that array anymore, e.g. if you do something like this.set('myArray', newArray);
In this case, I would like the initial array that myArray pointed to to get garbage collected. It's reference however is still contained in Polymer._collections which would prevent carbage collection.
Ist this handled by Polymer internally or is this a memory leak?
Copied from original issue: Polymer/docs#1698

Polymer._collections is a WeakMap so this alone shouldn't leak.

Polymer.Collection has been deprecated and removed, therefore I am closing this issue.


Update NativeWebview title only if the title changes

cody-agent-trace.json looks like this now:
[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

[Trace - 01:03:47 PM] Received notification 'webview/setTitle'
Params: {
  "handle": "native-webview-panel-df3675ed-b3d7-4051-94ff-78a569945443",
  "title": "Explain what @src/main/ja..."
}

I think there is no need to send the notification if the title does not change actually.
Test plan

Run IDE.
Run Explain a few times.
Review cody-agent-trace.json
The number of notifications should be reasonable.

Changelog

@sourcegraph/cody-agent 🙏
btw, I cannot see that consistent logs anymore (however the notifications still comes multiple times e.g. for explain code)


Add architecture diagram

Here is a good notebook example - https://github.com/IBM/pixiedust-facebook-analysis/blob/master/doc/source/images/architecture.png

Added


Question about database compatability

If I update our existing 2.0.0 AlphaFold databases to the AlphaFold databases for version 2.1.0, will the database files be compatible with AlphaFold 2.0.0? Or would I have to store two separate databases, one for users using v2.0.0 and the other for v2.1.0?
Thanks in advance for your help!

Yes, AlphaFold databases for version 2.1.0 will be compatible with AlphaFold 2.0.0 (instructions here for the update).
Also note that the AlphaFold 2.1.0 release includes the original AlphaFold monomer models (as a preset), so the users of the version 2.0.0 can likely be migrated to the newer version without any implications for their research.


Collecting credible sets (COJO outputs) never ends with whole seqid

The R script, workflow/scripts/s04_collect_credible_sets.R, combining COJO outputs containing the list of independent SNPs at each locus, is lazy and never ends while running the pipeline with list of all proteins sequences.

Solution: To avoid rule collect credible sets fails:

first scan all the files available at directory with cojo format,
then extract the seqid from the input sentinel files,
look for the paths containing the extracted seqids
read the COJO outputs using the path
merge all the data files.

In this way the job gets finished less than 5 minutes for 7K proteins.


Make it possible to confirm/apply popup action using keyboard

Currently popup messages could be only dismissed using keyboard. I would be good to have a way to apply affirmative action using keyboard as well.

Besides, from UX point of view 'Close' label could be misleading, as it is not always obvious that will be closed. Consider changing to 'Cancel'.

https://github.com/Microsoft/vscode/issues/5538


Fix bug in generating random kets in the unitary learning

In the example the random seed for the ket is fixed so in principle the ket's created are not really random. This must be fixed.

I think that the idea for the seed is to show a "pretty good" loss trajectory on a specific set of states. One may remove the seed. The results, however, will not be the same -- loss might be a bit higher.

The issue is that due to the seed you are not really creating random states
but the "same" state every time. So it's not making a bunch of states but
the same state. I will update it so that we set a seed before creating the
random states and therefore have a specific set of states as you say.
On Mon, 1 Feb 2021 at 11:13, Asad Raza<EMAIL_ADDRESS>wrote:

I think that the idea for the seed is to show a "pretty good" loss
trajectory on a specific set of states. One may remove the seed. The
results, however, will not be the same -- loss might be a bit higher.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/qgrad/qgrad/issues/75#issuecomment-770741417, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABVFIBH6MHXHSQIJEGJEPZDS4Z5GNANCNFSM4W3SWHEA
.

Good catch!
On Mon, Feb 1, 2021 at 10:22 AM Shahnawaz Ahmed<EMAIL_ADDRESS>wrote:

The issue is that due to the seed you are not really creating random states
but the "same" state every time. So it's not making a bunch of states but
the same state. I will update it so that we set a seed before creating the
random states and therefore have a specific set of states as you say.
On Mon, 1 Feb 2021 at 11:13, Asad Raza<EMAIL_ADDRESS>wrote:

I think that the idea for the seed is to show a "pretty good" loss
trajectory on a specific set of states. One may remove the seed. The
results, however, will not be the same -- loss might be a bit higher.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/qgrad/qgrad/issues/75#issuecomment-770741417, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ABVFIBH6MHXHSQIJEGJEPZDS4Z5GNANCNFSM4W3SWHEA
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/qgrad/qgrad/issues/75#issuecomment-770747009, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADPF67FILGHXH7UPYMFQXGDS4Z6IBANCNFSM4W3SWHEA
.

--
Dr. Nathan Shammah
www.nathanshammah.com


gtags-cscope job failed when opening extremely large project

I'm using gutentags with global only and it works very well for a long time.
However recently i'm trying to search the Linux Kernel source and get the error of gutentags: gutentags: gtags-cscope job failed, returned: 1 on startup.I tried another large project,the Unreal Engine and it fails again.
I manually run gtags in the root folder and it works,so it seems like a problem here. My related setting as follows:
"gutentags
let g:gutentags_project_root = ['.root', '.svn', '.git', '.hg', '.project']

let g:gutentags_ctags_tagfile = '.tags'

let g:gutentags_modules = []

let g:gutentags_modules += ['gtags_cscope']

let s:vim_tags = expand('~/.cache/tags')
let g:gutentags_cache_dir = s:vim_tags

let g:gutentags_ctags_extra_args = ['--fields=+niazS', '--extra=+q']
let g:gutentags_ctags_extra_args += ['--c++-kinds=+px']
let g:gutentags_ctags_extra_args += ['--c-kinds=+px']

if !isdirectory(s:vim_tags)
   silent! call mkdir(s:vim_tags, 'p')
endif

set cscopetag
set cscopeprg='gtags-cscope'

Hi! Try pulling the latest Gutentags code, and :let g:gutentags_trace = 1 to get some debug logging. For instance, it should show you the gtags command-line that it used to run, so you can compare how Gutentags runs gtags with how you ran it yourself in the project root.


test: add internal package test case

Background
Currently, grm has no test cases, we should supplementary test cases
Description

Others

cc @XeryYue

you are right. but can we  validate the test at ci？

you are right. but can we validate the test at ci？

Fine, I would like to do this work.


Show Album Cover command not working in full music player

OS: macOS Sierra 10.12.5
Issue Descriptions: Show Album Cover command not working in full music player
Steps to Reproduce: While not in mini player, verify none of the following causes the album cover to appear or disappear (album cover expected to take up entire view).

Click the album artwork in the bottom left.
Hit the Shift+Command+C hotkey to show album cover.
Go to View, click Show Album Cover.

I think there used to be white arrows that appeared when hovering over the mini album artwork, if so I don't see them now.
Great app, thanks for the hard work!
debug_info.zip

Google has removed the album art visualizer as well as the particle effects visualizer and recent updates. We may be able to emulate the album art static viewer in future updates.

We'll add it to the to-do list.


Remove unimplemented configuration properties

Related to #520
Motivation:

We need to remove or implement unimplemented configuration properties.

Modifications:

Remove maxPendingRequests configuration property from ServerConfig and ServerBuilder classes.
Add MaxConnectionHandler to limit the number of open connections.
Change DEFAULT_MAX_CONNECTIONS from 65536 to Integer.MAX_VALUE in order not to limit open connections by default.

Result:

maxPendingRequests configuration property is removed.
We can limit the number of open connections to the configured value.

I think it's better to implement maxConnections property than removing the property, so I added MaxConnectionHandler. Please let me know if you have a better idea. :)

TODO: Think about the way to lower logging pressure.

@trustin I changed the codes about writing a log message. PTAL the below link.
Log writing job will be scheduled in boss's event loop, because the child channel doesn't have its event loop at the point. The child channel is not registered yet. Is it okay?
https://github.com/line/armeria/pull/527/commits/7a06817e68e1e9adb12723b98b29d03710725e90#diff-38497fb6ffd9735e61fe2ee7e1c5b585R73

@trustin Done! PTAL if you have time. :)


Update rococo registration

Instead of using a release tag, explain how to get the present commit to use in local testing based on public RPC node's version (exposed in JS app, js console examples)

explain:

use the live chainspec: https://github.com/paritytech/polkadot/tree/master/node/service/res
set correctly on template chain_spec.rs
PR to add apps UI to include your network in the list (https://github.com/polkadot-js/apps/tree/master/packages/apps-config)


🌳🪄🌱Create smaller Trees on Serialization

fixes #94

maybe there'll be some more cases to consider one day but it shouldn't be to difficult to add them later. the hard part is keeping it deserializable which we don't need here.


Changing the cluster domain

I can't find any documentation on how to change the cluster domain. From looking around the system applications in the Flynn dashboard I assume that I'd have to change it in all the configurations and routes there, but are there any other places where the domain might be configured?
I'd like to avoid reinstalling because of all the work that already went into setting things up. Any advice on how to do this would be greatly appreciated :)

flynn cluster migrate-domain example.com will do a domain migration, including updating routes, please be sure to take a backup first and ensure the DNS is configured properly before starting.

Thank you! I guess I must be blind not finding that.

Hi @titanous
I'm in a similar situation where I'm trying to recreate my production environment on a different hosting.
I've made a backup of my current cluster, and restored it on the new enviroment using the bootstrap command with a different domain name. But when I try to get the cli-add-command from flynn-host the domain name returned is not the new one.
Is there a way to change this?
Thanks,
Salvo


[TASK] Replace sudo with become because of deprecation

Sorry, I mixed up both sudo and becom, but all should work ok


SPARKTA-393 fix CustomExceptionHandler behaviour

Removed implicit handler from some classes in order to use expected CustomExceptionHandler.

The exception handler is removed, we don't need this?

We don't need implicit handler within controllers.  We've already got a
CustomExceptionHandler class that we were not using.
On Tue, Feb 16, 2016, 12:25 AM Jose Carlos Garcia Serrano <
<EMAIL_ADDRESS>wrote:

The exception handler is removed, we don't need this?
—
Reply to this email directly or view it on GitHub
https://github.com/Stratio/Sparta/pull/1399#issuecomment-184439320.

--
Emilio Ambrosio Cordero | Big Data Developer
@_eambrosioc
Vía de las Dos Castillas, 33, Ática 4, 3ª Planta.
28224 Pozuelo de Alarcón. Madrid.
+34 918286473 // @stratiobd https://twitter.com/StratioBD

LGTM

All ok


Avoid more NPEs and handle version= in export declaration

Patch includes more additional NPE checking/traping in the case there's no exports packages.
And updates the parser to return with an EXPORT rather than ERROR if the last character in the directive is a ".

The more complex conditions in the state transitions you added made me suspicious and indeed there was another erroneous state transition that made your changes unnecessary.
I therefore reverted your changes in the parser and applied my fix. But thank you very much for all the work you put into this! The unit tests you added helped me a lot.

@metlos oh cool! Glad the tests helped at least :)


Remove hackish and not multiplayer ready crouch visuals.

Contains
This removes the constant camera up/down movement in multiplayer when you crouch.
It also makes it so that while "default croaching" is enabled with key "x", presseing the temporary croach button "ctrl" will make the character temporarly stop crouching. This is analog to how it is for running.
This change removes all the hackish existing player height and eye height modificaitons during crouching. The old code had the following issues:

the client ordered the server to change the eye height based on possibly outdated eye height values it had received previously from the server
character height and eye height change got triggered by client instead of being a result of movement mode change
client triggered redundant movement mode changes that would also
automatically result from movement input
movementDebugCommands were used to do eye height and height modification. The commands are hackish too and not really something that should be used in non debug code.
the player height change triggered a teleportation of the player which messes up further with the movement prediction system

How to test
Start a headless server and join as client. Then try crouching. Movement speed should be slightly slower while crouching.

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

I do agree that most of the code was 'hacky' and not ready for multiplayer. However, this change simply destroys the point of "crouching" which is reduction of height. The shift (Run/Walk toggle) button probably would do the exact same thing as crouching now.
If you want to remove this for the sake of removing the multiplayer bug or if you plan to initiate a movement refactoring rework soon, I don't mind going ahead with this. Otherwise, it is a step in the opposite direction and simply undoes things.

I think the main intention of crouching was to have a "do not fall of edge" feature which was not implemented yet anyway.
The size reduction is not something that will be needed often / at all.
The current code just solves it in the wrong way. Having it there just gives other developers bad examples of how to do stuff.

I agree the code was problematic in multiplayer, but it did work really well in single player. The crouching for height differences is also desirable imho :-)
Could we live with a workaround for multiplayer (disable crouching entirely if needed) along with some big visible TODOs while keeping it functional as-is in single player? Or is that too "code smelly" ? I don't know who might have time to finish this since it isn't part of GSOC which will eat most of our time.
Or would it be better to park the code elsewhere entirely and wrap it into the overall movement refactoring whenever we get somebody active on that area?

I think crouching actually allows you to pass through 1x1 areas (like if we had vents or small tunnels). It would be useful if we ever wanted to make adventure maps with secrets, traps or w/e else.
Completely removing it for now is ok, but the idea is good and we should get back to it when possible.

Hooray Jenkins reported success with all tests good!

Could be a really cool issue to work on right after GSOC. Movement refactoring is a bigger topic, so we could try to get students, mentors, and anybody interested work together on this right after the coding period ended, to not loose the focus and drive we just gained.

Seeing as it's raising bugs like #3108 I think it would be good to merge this and then make it some kind of gsoc task

Hooray Jenkins reported success with all tests good!

Fixed the conflict and gave it a short test.
This change is more like a sneak than a crouch but it works and my personal bias is to have a working feature with clean code rather than a broken one with more bad code behind.
I suggest to merge this PR, then close #2555, #2651, #3108 and continue on #1075 (it is the only one with a bounty) with the requests from the named issues.
Full list for crouching for me would be:

Multiplayer support (host & connected client)
1x1 holes enterable
Visual Feedback in eye height
Falling of ledges support
Toggle/Untoggle + temporary button (as is with this PR)
Leave 'LocalPlayerSystem' cleaner instead of making it worse.

For the last point it would make sense to clean up the system first (after this PR) which is something i can work on.
I will leave this PR open until next weekend for extra feedback (ping @Cervator @jellysnake @skaldarnar @nihal111 @flo) then merge and continue as proposed if no one raises an objection 😉

Left it open a bit longer :)
I will now update the issues. The follow-up will be #1075.

Thanks for merging


Initialize active camera object to get frame fast

Hey,
I was wondering, what would be the correct way of initializing some sort of singleton for a camera object, that could be called to get frames very fast? Something like:
CAMERA: Camera = None // lazy optional struct initialized via initialize_camera?

fn initialize_camera() -> Result<(), String> {}  // to execute somewhere, at the beginning of the execution

fn capture() -> Result<Vec<u8>, String> {}  // to execute everytime we need a frame

Could someone put me on the right track for the definition of Camera? and where is the best place to read documentation that would help resolving this?

Hi,
I think the easiest way would be to put all the code from example (i.e. https://github.com/lit-robotics/libcamera-rs/blob/main/libcamera/examples/jpeg_capture.rs) into a thread and communicate over channels. You would need 2 channels, one to send a signal to capture and another to wait for response. The camera thread would loop on waiting for signal and then send an actual capture request to camera. Once request is completed, it would return the result over the second channel.
So your capture() function would just send a signal to a channel to start capture, and then wait on the second channel for the response.
Let me know if this is clear or if you need additional help.


✨(dimail) automate allows requests to dimail when creating domain accesses

Purpose
Automatically send user creation and allow creation requests to dimail
upon creating domain access.
Proposal
Description...

[x] client function to send user creation request
[x] client fuction to send allow creation request
[x] test basic case for admin/owner
[x] test basic case for viewer (no call made)
[ ] test wen user already exists
[ ] make sure it triggers when creating a domain !
[ ] handle update of existing domain access (in another PR ?)

Something it breaking test test_api_mail_domain__accesses_create_authenticated_administrator : the validate method should raise an exception line 177 and prevent us from entering the create. Still, it seems like it doesn't stop us.
Do you have any idea what is happening, @sdemagny ?

@qbey e2e tests are failing because of a navigation action that doesn't work as expected. Could it be due to the fact that, upon logging on localhost:3000 as people@people, I land on /teams/ and mail-domains tab icon has disappeared ?
I feel like you mentioned something like that the other day.

Reproduction en local de ce qui casse le test e2e :


Add unit test for scan command

Added some rudimentary tests for the scan command, mocking away the file io.

run the test using go test ./scan

Hi this, I had no time yet to review this.
I started this morning but I don't understand some different things and choices, so I will take a couple of minutes more to be sure to have a proper look.

Hi @bygui86, I am going to go ahead and merge this PR because I want to continue working on the unit tests. I suggest that we do a review of all of the test code when I am done.

Hi @mkretz thanks you for taking care of it.
This period is really busy for me. I'm not able to follow PRs and development of the project for now.
Can I still be included for a while? I really would like to continue contributing, but for now I have to think about my wedding and my new job.

Can I still be included for a while? I really would like to continue contributing, but for now I have to think about my wedding and my new job.

Sure thing, will keep you in the loop! Thanks for your contributions so far! Those are indeed important topics :-)! Congrats on both!


Instructor should be able to add tools to their course navigation

please track further development of add/remove LTI tool feature in the #21


Disabling Email Verification option

Is your feature request related to a problem? Please describe.
When installed locally in a cloud environment like AWS, outgoing emails cannot be set over port 25 since AWS blocks this port.
Describe the solution you'd like
Option to disabled email verification or ability to just create local user accounts

Workaround
Not the solution, but, in the meantime, you can just see logs from the oneuptime-mail-1 container with:
docker logs oneuptime-mail-1

and then see the verification link for the user in the log, for example:
{
  message: 'Request ID: 7a4d2aa0-ac1d-11ed-8a9a-99b27d1a1a27 -- POD NAME: NONE -- METHOD: POST -- URL: /email/send\n' +
    ' Request ID: 7a4d2aa0-ac1d-11ed-8a9a-99b27d1a1a27 -- Request Body: {\n' +
    '  "toEmail": "test_user@example.xyz",\n' +
    '  "subject": "Welcome to OneUptime. Please verify your email.",\n' +
    '  "templateType": "SignupWelcomeEmail.hbs",\n' +
    '  "vars": {\n' +
    '    "name": "Test User",\n' +
    '    "tokenVerifyUrl": "https://oneuptime.xyz/accounts/verify-email/7a400b40-ac1d-11ed-b775-8f755056301b",\n' +
    '    "homeUrl": "https://oneuptime.xyz/"\n' +
    '  },\n' +
    '  "clusterKey": "82f98a4bce16a09978c7cacf"\n' +
    '}',
  level: 'info',
  timestamp: '2023-02-14T04:10:12.427Z'
}

I believe this was fixed in https://github.com/OneUptime/oneuptime/releases/tag/7.0.645.

Closing this issue.


changed location

Sublime put all packages into a user folder now. Other paths need updating

Should have both for people running <3061 and those running >=3061.


[Bug] When I press ”Add a new style...” some icons remain on the screen

Describe the bug
When I press ”Add a new style...” some icons remain on the screen
Browser
Brave [Version 1.37.116 Chromium: 100.0.4896.127 (Official Build) (64-bit)]
Windows 11 Pro
To Reproduce
Steps to reproduce the behavior:

Go to Styles
Click on ”Add a new style...”

Expected behavior
I should've seen clearly the Enter URL and Code blocks.
Screenshots

This is now fixed as part of #703


Support plugin for Vert.x

Please answer these questions before submitting your issue.

Why do you submit this issue?
[x] Question or discussion
[ ] Bug
[ ] Requirement
[ ] Feature or performance improvement

Question

What do you want to know?

I am creating a plugin for supporting Vert.x powered applications using SkyWalking. Everything appears to work when I spread out the events so they each happen one after the other but once I start letting the messages flow naturally SkyWalking won't finish a single span.
For example:

I call GET on URL /products/test (this creates an ExitSpan)
I receive GET on URL /products/test (this creates an EntrySpan)
I respond to GET on URL /products/test (this closes the EntrySpan)
I receive response for GET on URL /products/test (this closes the ExitSpan)

The above work correctly and the SkyWalking UI appears to agree. It's when the flow of events changes that SkyWalking will not finish a trace.
For example:

I call GET on URL /products/test (this creates an ExitSpan)
I call GET on URL /products/test (this creates an ExitSpan, child of previous)
I receive GET on URL /products/test (this creates an EntrySpan, child of previous)
I receive GET on URL /products/test (this creates an EntrySpan, child of previous)
I respond to GET on URL /products/test (this closes the EntrySpan, child of previous)
I respond to GET on URL /products/test (this closes the EntrySpan, child of previous)
I receive response for GET on URL /products/test (this closes the ExitSpan, child of previous)
I receive response for GET on URL /products/test (this closes the ExitSpan)

In the above example, SkyWalking would eventually finish a trace but it would be the single trace instead of the expected two traces. The reason I say that SkyWalking never finishes a trace is that I have the above occurring every second and since SkyWalking keeps adding spans as the child of the previous it keeps a non-empty span stack and continues without finishing any traces.
From my best understanding of what's happening, the TracingContext class requires either of two scenarios to create a new trace:

TracingContext.activeSpanStack is empty
The current thread hasn't started a trace

If that's correct then I don't see how I can get this to work with Vert.x. In this case, Vert.x is using the same thread to run all of the events on. Both of the GET requests, both of the GET receives, and both of the GET responses happen on the same thread and I can't simply use the InstanceMethodsAroundInterceptor.afterMethod method to close spans because of the asynchronous nature of Vert.x.
If my understanding is correct what would you guys recommend to tackle this issue?
Which version of SkyWalking, OS and JRE?
SkyWalking 6.0.0 GA, Ubuntu 18.04, Java 8

If my understanding is correct what would you guys recommend to tackle this issue?

I think most of your understanding is correct, the thread model of vert.x is a typical model in RPC framework. Also, SkyWalking did request the TracingContext working in that way, most of tracing system asked for this too.
The key is, you created the span in a shared thread(that single thread, I am guessing this is nio/vertx event loop thread).  SkyWalking is focusing on business logic, the span could be created at the entrance of business codes, such as we did for SpringMVC. We create span at Controller edge, then, no matter which web container is used inside, the span will work as expected.
Like in the codes
public class MyServer extends AbstractVerticle {

    @Override
    public void start() {

        Router router = Router.router(vertx);

        router.route().handler(this::queryHandler);
        vertx.createHttpServer().requestHandler(router::accept)
                .listen(8080);
        System.out.println("MyServer startup");
    }

    private void queryHandler(RoutingContext routingContext) {
        routingContext.response()
                .putHeader("content_type", "application/json")
                .end("hello world");
    }

Span should be created when #requestHandler called, there should not be a shared thread at this point, because it can't in Java thread model, the requestHandler will hold the thread until it is done or start a child thread(that is context capture/continue designed for)

I had a lightbulb moment when you said the key was I was creating spans on a shared thread. I'm pretty sure you're absolutely right. Unfortunately, I think that's the only way to use Vert.x. Everything that runs in the verticle is ran on the event loop and in my situation there is only a single event loop. That's what I'm seeing locally as I debug the creation of the spans. The thread is always vert.x-eventloop-thread-0.
I should have included some Vert.x code to begin with to open this discussion. I've included some below and described the steps the code takes and the steps I take to trace it.
public class Sender extends AbstractVerticle {

    public static void main(String[] args) {
        Vertx vertx = Vertx.vertx();
        vertx.deployVerticle(new Sender());
    }

    @Override
    public void start() {
        Router router = Router.router(vertx);
        router.route().handler(BodyHandler.create());
        router.get("/products/:productID").handler(this::handleGetProduct);
        vertx.createHttpServer().requestHandler(router).listen(8080);

        //vertx.setPeriodic(1000, v -> callGetProduct());
        callGetProduct();
    }

    private void callGetProduct() {
        HttpClient httpClient = vertx.createHttpClient();
        httpClient.getNow(8080, "localhost", "/products/test",
                new Handler<HttpClientResponse>() {

                    @Override
                    public void handle(HttpClientResponse httpClientResponse) {
                        System.out.println("Got response");
                    }
                });
    }

    private void handleGetProduct(RoutingContext routingContext) {
        HttpServerResponse response = routingContext.response();
        vertx.setTimer(4000, it -> {
            response.setStatusCode(200).end("Done");
        });
    }
}

I've tried to keep it as simple as possible and I think I'm creating the right spans at the right places.

I have an HttpClient which calls a GET request on /products/test. This opens a ExitSpan.
This request is received by handleGetProduct. This opens a EntrySpan.
The response is delayed by 4 seconds and then responds with "Done". This closes the EntrySpan.
The response is received by handle. This closes the ExitSpan.

The locations I'm doing the opening and closing of spans seems correct to me as well.

The HttpClient sends the GET request by calling HttpClientRequestImpl.end(). I take the ContextCarrier and inject it into the header and open the ExitSpan. I also assign the ContextSnapshot to the request (via EnhancedInstance) for closing later.
The GET request is receieved and sent to the target method via HttpServerRequestImpl.doEnd(). I remove the headers to rebuild the ContextCarrier and open a EntrySpan. I also assign the ContextSnapshot to the response (via EnhancedInstance) for closing later.
4 seconds pass
I pull the ContextSnapshot from the response (via EnhancedInstance) and close the EntrySpan once HttpServerResponseImpl.end() is called as this signifies the response has been written.
I pull the ContextSnapshot from the request (via EnhancedInstance) and close the ExitSpan once HttpClientRequestImpl.doHandleResponse() is called as this signifies a request has been replied to.

Again, this all works when I just go through this flow once. The single call in the code to callGetProduct(); fulfills this scenario. It's once //vertx.setPeriodic(1000, v -> callGetProduct()); is uncommented that it starts acting up. As you can see, setPeriodic makes the call to callGetProduct every second. This means that I won't have the previous trace closed by the time the next one starts.
Given that Vert.x is running everything on a shared thread, and the goal is to not create spans on a shared thread, what's my move here?

The HttpClient sends the GET request by calling HttpClientRequestImpl.end(). I take the ContextCarrier and inject it into the header and open the ExitSpan. I also assign the ContextSnapshot to the request (via EnhancedInstance) for closing later.

You should finish the span, and hold the ContextSnapshot. When callbacke happens, create a new local span(should be new context, because other closed their exit span too), and continue the ContextSnapshot.

I pull the ContextSnapshot from the response (via EnhancedInstance) and close the EntrySpan once HttpServerResponseImpl.end() is called as this signifies the response has been written.

From you description, I am a little confused about how you use ContextSnapshot and ContextCarrier. How does I pull the ContextSnapshot from the response happen? The ContextCarrier is designed for across processes(RPC, MQ), ContextSnapshot is used across thread in a single process only. Is this just typo or do I miss somthing?

I pull the ContextSnapshot from the request (via EnhancedInstance) and close the ExitSpan once HttpClientRequestImpl.doHandleResponse() is called as this signifies a request has been replied to.

In pure async scenario, like vertx, response is just a new local span, entry span should be closed as soon as an async jos assignment happens. Don't wait until this reponse called. Because if the thread model you provided, the thread could do anything because another thread to process the logic and do the response.

It's once //vertx.setPeriodic(1000, v -> callGetProduct()); is uncommented that it starts acting up. As you can see, setPeriodic makes the call to callGetProduct every second. This means that I won't have the previous trace closed by the time the next one starts.

I think my prev explanation about when is the right time to close span, should make this easier. Create snapshot and finish the ongoing span when do setPeriodic, then create a local span when v -> callGetProduct happens.

I think my most answers focus on one thing, close/finish span before async job happens. :) Hope this would help you.

@wu-sheng, your advice appears to have gotten me closer but I think there is something I'm fundamentally not understanding. I able to produce the following trace stack:
{
  "traceSpans" : [ {
    "traceId" : "7.39.15523638435290001",
    "segmentId" : "7.39.15523638435290000",
    "spanId" : 0,
    "parentSpanId" : -1,
    "refs" : [ ],
    "serviceCode" : "8fd2ffff-8d5b-46ed-a681-951fb18ed1d1",
    "startTime" :<PHONE_NUMBER>533,
    "endTime" :<PHONE_NUMBER>595,
    "endpointName" : "/products/test",
    "type" : "Exit",
    "peer" : "/products/test",
    "component" : "Vertx",
    "layer" : "RPCFramework",
    "tags" : { },
    "logs" : [ ],
    "error" : false
  }, {
    "traceId" : "7.39.15523638435290001",
    "segmentId" : "7.39.15523638478920006",
    "spanId" : 0,
    "parentSpanId" : -1,
    "refs" : [ {
      "traceId" : "7.39.15523638435290001",
      "parentSegmentId" : "7.39.15523638435290000",
      "parentSpanId" : 0,
      "type" : "CROSS_THREAD"
    } ],
    "serviceCode" : "8fd2ffff-8d5b-46ed-a681-951fb18ed1d1",
    "startTime" :<PHONE_NUMBER>892,
    "endTime" :<PHONE_NUMBER>892,
    "endpointName" : "HttpClientRequestImplDoHandleResponseInterceptor-#/products/test",
    "type" : "Local",
    "peer" : "",
    "component" : "",
    "layer" : "Unknown",
    "tags" : { },
    "logs" : [ ],
    "error" : false
  }, {
    "traceId" : "7.39.15523638435290001",
    "segmentId" : "7.39.15523638438680002",
    "spanId" : 0,
    "parentSpanId" : -1,
    "refs" : [ {
      "traceId" : "7.39.15523638435290001",
      "parentSegmentId" : "7.39.15523638435290000",
      "parentSpanId" : 0,
      "type" : "CROSS_PROCESS"
    } ],
    "serviceCode" : "8fd2ffff-8d5b-46ed-a681-951fb18ed1d1",
    "startTime" :<PHONE_NUMBER>869,
    "endTime" :<PHONE_NUMBER>870,
    "endpointName" : "/products/test",
    "type" : "Entry",
    "peer" : "",
    "component" : "Vertx",
    "layer" : "RPCFramework",
    "tags" : { },
    "logs" : [ ],
    "error" : false
  }, {
    "traceId" : "7.39.15523638435290001",
    "segmentId" : "7.39.15523638478710004",
    "spanId" : 0,
    "parentSpanId" : -1,
    "refs" : [ {
      "traceId" : "7.39.15523638435290001",
      "parentSegmentId" : "7.39.15523638438680002",
      "parentSpanId" : 0,
      "type" : "CROSS_THREAD"
    } ],
    "serviceCode" : "8fd2ffff-8d5b-46ed-a681-951fb18ed1d1",
    "startTime" :<PHONE_NUMBER>871,
    "endTime" :<PHONE_NUMBER>882,
    "endpointName" : "HttpServerResponseImplEndInterceptor-#/products/test",
    "type" : "Local",
    "peer" : "",
    "component" : "Vertx",
    "layer" : "Unknown",
    "tags" : { },
    "logs" : [ ],
    "error" : false
  } ],
  "total" : 4
}

This produces the diagram:

Which is sort of odd when it considers the same trace as:

It seems to somewhat understand that it took 4 seconds to complete.
Essentially I followed your advice and made the process as such:

When the GET request is called the ExitSpan is opened on beforeMethod and the ContextCarrier is added to the header. The ContextSnapshot is saved to the SkyWalkingDynamicField for the request. The ExitSpan is closed on afterMethod.
The GET call is received and a EntrySpan is opened with the ContextCarrier extracted from the request's headers during beforeMethod. The ContextSnapshot is set on the SkyWalkingDynamicField of the response object.  The EntrySpan is closed on the afterMethod.
4 seconds pass
HttpServerResponseImplEndInterceptor.beforeMethod is triggered and creates a LocalSpan and calls ContextManager.continued with the ContextSnapshot saved to the response object. The LocalSpan is closed on the afterMethod.
HttpClientRequestImplDoHandleResponseInterceptor.beforeMethod is triggered and creates a LocalSpan and calls ContextManager.continued with the ContextSnapshot saved to the request object. The LocalSpan is closed on the afterMethod.

I don't exactly have a question here so I hope you see what I'm doing wrong. The code I'm testing with is located here: https://github.com/apache/incubator-skywalking/compare/master...BFergerson:master

Essentially I followed your advice and made the process as such:

The processes you mentioned make sense to me, it should be processed in this way when you are facing async scenarios. I assume right now, the tracing and metric should be working well even in concurrency cases, right?

Just one suggestion, the local span you created, the operation name should be related to target framework behaviours, like a description, rather than which interceptor of SkyWalking plugin is doing this. Because in most cases, interceptor is just being known by plugin developer.
Of course, maybe this is just because, you are testing. :) If so, ignore me.

@wu-sheng, yeah those operation names are for testing right now. I've deleted code to try to keep what I'm showing you as straight forward as possible. I'll make sure to address that once everything is working.
And yes, your suggestions do fix the problem I was originally having with the traces becoming children of each other and never sending any to SkyWalking, but now I'm dealing with this:

As you can see, it doesn't correctly understand that /products/test took 4 seconds to respond. It understands it when you open the trace up but on the endpoint dashboard it doesn't. What causes this discrepancy? Do the trace spans I'm producing in the JSON above seem correct?

As you can see, it doesn't correctly understand that /products/test took 4 seconds to respond. It understands it when you open the trace up but on the endpoint dashboard it doesn't. What causes this discrepancy? Do the trace spans I'm producing in the JSON above seem correct?

Actually, this is what I expect. Because the segment duration is shorter, the endpoint duration is based on that.
Also, if you want to show the segment perfectly, we need to build a new TracingContext implementation, with clear explicit declare to use it. In my previous experiences, that style of APIs is not very friendly to plugin developer, so I haven't implemented it.  If you want to take part in more, I could start a draft and let us work on that.

@wu-sheng, I would definitely like to implement support for Vert.x so if it's necessary to create a new TracingContext implementation to do that then yes, please start a draft.

Because the segment duration is shorter, the endpoint duration is based on that.

Before you do would you mind breaking down a bit why a new implementation is necessary? Specifically the logic quoted above. I understand a span to be a single "unit of work". From the SkyWalking documentation a segment is "a concept in SkyWalking, it should include all span for per request in a single OS process, usually single thread based on language". Based on these definitions, wouldn't everything involved with finishing the request for /products/test be included in the segment? Meaning why not add logic which indicates that a segment is always equal to or longer than the duration of the spans which it consists of?

Sorry for don't make this clear to me.
a segment is "a concept in SkyWalking, it should include all span for per request in a single OS process, usually single thread based on language.
The Key in Java for plugin developer convenience, segment always works in a single thread at the same time. And this fits most of scenarios in Java codes. But in vertx(react style), things changed.
Your case is very typical.
Thread accept the request, then

Create span, meaning, create a segment.
Try to assign the work to another thread
Right now, the thread still holds the segment/tracingcontext in thread local.

That is why if we don't finish that, the new incoming request uses the existing trace context, which triggers your issue at the beginning.
But we finish the span, then because tracing context is a stack-based structure, right? The tracing context finished, and segment finished too. So at this moment, the thread local is available for new request. That is why you traces look like working well in concurrency requests.
The side-effect of doing this, is segment finished when your work assignment is finished, not works are done because that is in another thread, right?
SkyWalking backend analysis the response time based on segment and entry span, that is why you can't see 4s(in your case) here, but you can see 4s in trace visualization. The trace visualization shows 4s because UI did that.
SkyWalking analysis 100% trust the entry span and segment, that is why we are high performance and don't need distributed cache in any case. So when request entry span says ~0s duration, the endpoint metric show in the same way.

I am proposal the complex tracing context implementation is

TracingContext and Segment could go across threads in Java
Span finished time could be reset in another thread, because of (1), you could get that, and context would not finish before you say so.

But in this case, API of context is more complex, and lock mechanism must be involved.

@wu-sheng, I think I'm almost on the same page with you. You're saying the JSON trace stack I included above is correct in all aspects except for the endTime on the ExitSpan and EntrySpan? And it's because of the fact that the EntrySpan has an incorrect endTime that the response time shows up as it does in the UI? And finally, this can't be done with the existing TracingContext because it will finish the current Segment and send that to the collector before the callback happens (so no possibility for modifying the endTime.

TracingContext and Segment could go across threads in Java

If my understanding is correct then how does this proposal work? The TracingContext would have already finished the Segment even if I serialize/deserialize them at time of the callback? Would there be additional logic in this TracingContext implementation which prevented it from finishing a Segment of just one EntrySpan?
Thanks for all the help BTW. I'm learning a lot.

Yes. In my mind, for new tracing context mechanism(only optional), if you do capture, it will hold the segment untill continue happens. Only finish the whole segment and tracing context, if and only if all related threads are finished and no capture without continued.
Does this make sense?

@wu-sheng, I think I have a firm grasp of the problem now lol. Instead of modifying the endTime why not just wait to close the span? I understand that waiting to close the span won't work with TracingContext but that's because it deals with one segment at a time. A new TracingContext implementation could work with multiple segments at a time requiring a key to choose which segment the spans belong to. I think with an implementation like this I could delay closing the EntrySpan until the response is sent just like I was before. That also lets the process for all the TracingContext implementations be essentially the same and this multiple segments/asynchronous one doesn't have extra look back logic.

In theory yes, just delay to close the span, in practice way, it is hard. Because the span should still in stack-based structure, as a plugin developer, you can't tell whether there is one/more parent span(s) need to close by other plugins, right? So you must follow the stack order, to close the span. That mean, entry span must be finished first and reset finish time later.

Edit: Also, I could see how the same could be implemented with multiple TracingContext instead of an implementation with multiple segments. I'm thinking something like AsyncContextManager.

Are you saying to use AsyncContextManager replace ContextManager? If so, I think that will not work. There are plenty of plugins work out there using ContextManager as core API, you change that, means propagation context in the thread with other plugins will fail. I think AbstractTracerContext implementation makes more sense.

@wu-sheng, okay the ContextManager implementation might be too far. How is this not an issue for other async plugins you have though (eg: Apache httpcomponent AsyncClient 4.x)? Also, what kind of draft would you be starting?

Edit: You're probably going to say you don't start/close spans in the shared thread :P

Yes and there is a little different for them. Because they just do async call once for a time, and for their developer, they think, it just blocks short time, then the short duration makes sense. So they put callback in another thread and another segment/tracingcontext just like you did.
The ContextManager implementation is not hard, just it must count on continue must be executed after you call capture. Otherwise, the context would never finish and span in first context will be totally lost.
Does this make sense to you?

AcrossThreadContextTracing should be the right term for this.

If it works, we could make asynchttp, grpc-streaming work in this way too. And in .net too.
cc @liuhaoyang @ascrutae @hanahmily I think you may be interested in this term and design too.

@wu-sheng, I guess I'm just not entirely sure where you see the difficulty in holding the EntrySpan open until the response. Even in a completely asynchronous environment, the response will be the last event to occur. The EntrySpan is opened, all the various callbacks involved are handled through LocalSpan objects which add to the appropriate segment (stack-based all the same), and then when the response is finally ready to be written the EntrySpan is retrieved, closed, and the segment is flushed to the collector.
Why would I need to know what the parent span is? Each span is dealt with individually. If I open a span I have to make sure to close it. If that happens then by the time the response is ready the active span will be the correct span to close.
Where is the difficulty that you're seeing that I'm not?

Why would I need to know what the parent span is? Each span is dealt with individually. If I open a span I have to make sure to close it. If that happens then by the time the response is ready the active span will be the correct span to close.

First of all, there isn't about you need to know the parent span. It is more about when tracing context could finish. But in SkyWalking design and its API, you could notice that we don't have an API called TracingContext#finished from any plugin, just inside TracingContext triggered automatically, WHY?, because no plugin knows he is the last/first span.
In codes level, read TracingContext#activeSpanStack and EntrySpan(also its parent StackBasedTracingSpan), you will find out the span finish operation need to do in the exact same order as open it. Also TracingContext is in the ThreadLocal, which ones, one thread could only have one TracingContext.
Based on this background, how do you don't finish the entry span but switch the TracingContext to another request?
So, things like EntrySpan is retrieved only could happen when tracing context doesn't finish even all spans finished(removed from the active stack). That is key I mean for new TracingContext implementation, hold the context and span, make you have the chance to change/adjust the span.
Does this make more sense to you now?
This is a very complex scenario. Argue and the trade-off between development convenient and API capabilities. Most tracing systems ignore this to do based on Dapper paper says, so backend will face heavy payload. But they, like Zipkin and Jaeger, are not APM solution, so this doesn't matter for them. In another way, this design is common in APM vendor core, which they keep in the shadow :)

Another way to do a thing is by providing a new Entry/Exit Span feature to support EntrySpan is retrieved. Such as, Span#finishWithFurtherNotitify, which remove it from the active span, but still hold the tracing context. Only when Span#finishcalled(in another thread), tracing context finished and sent.

@wu-sheng, I think I understand. I assumed that the Vert.x EntrySpan would be the root one. I wasn't thinking bigger picture. I agree about modifying the endTime after the fact.

I will try to start a draft of my last proposal, provide span a feature for further explicit finish notification, which could be done in another thread. If I find other conflicts, I will inform you too.

PR submit. A draft version

@wu-sheng, using the new asynchronous functionality I'm able to get the following results:

This appears correct to me but I'm able to achieve these results without creating local spans and without using ContextSnapshot. I simply call prepareForAsync at the beginning of the request and asyncFinish at the end of the request. Same situation with the response. This produces the results above with the single exit span and single entry span.
Is that you meant for it to be used? Previously you said that the callback was supposed to be a new local span. Is that still the case when using the asynchronous functionality? If so, what is the order of events I should be taking. First open the local span, then close the entry span via asyncFinish then close the local span?

Is that still the case when using the asynchronous functionality? If so, what is the order of events I should be taking? First open the local span, then close the entry span via asyncFinish then close the local span?

From the trace, you should be using APIs in the right way. The other steps are about, you should create new context(including local span) in another thread(callback) in case there is other further plugin working.
Such as in your codes
    private void callGetProduct() {
        HttpClient httpClient = vertx.createHttpClient();
        httpClient.getNow(8080, "localhost", "/products/test",
                new Handler<HttpClientResponse>() {

                    @Override
                    public void handle(HttpClientResponse httpClientResponse) {
                        /////////////////////////////
                        // Add more real codes here,
                        // including call another RPC or access Database
                        /////////////////////////////
                    }

You would like the trace could continue in #handle method, right? If you don't create new context in handle thread, the trace will break, because async span doesn't include the capability to propagate context. Async span is just for making sure duration/tag/log/etc. fields right/accurate.

I am glad we are almost done:)

@wu-sheng, everything makes sense when you say it :P. I've added the local span back. I think the last question I have before I repeat this pattern for other Vert.x classes is:

does the order of ContextManager.continued and asyncFinish make any difference?

I've tried both and they both work and seem to produce the same traces.
The way I have it currently coded for the callback method is:
@Override
public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class<?>[] argumentsTypes,
                         MethodInterceptResult result) throws Throwable {
    VertxContext context = (VertxContext) objInst.getSkyWalkingDynamicField();
    ContextManager.createLocalSpan(context.getContextSnapshot().getParentOperationName());
    ContextManager.continued(context.getContextSnapshot());
    context.getSpan().asyncFinish();
}

@Override
public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class<?>[] argumentsTypes,
                          Object ret) throws Throwable {
    ContextManager.stopSpan();
    return ret;
}

The above code is the interceptor for the method:
@Override
public void handle(HttpClientResponse httpClientResponse) {
    /////////////////////////////
    // Add more real codes here,
    // including call another RPC or access Database
    /////////////////////////////
}

does the order of ContextManager.continued and asyncFinish make any difference?

Nothing related.

PR merged. When you are ready, you could submit the vert.x PR :)

there is work for tracing done in Vert.x for 4.0 , it is described in this spec: https://github.com/vert-x3/wiki/wiki/RFC:-context-tracing
also there are two implementations with OpenTracing and Zipkin here that can be used as examples https://github.com/vietj/vertx-tracing

This is an auto instrumentation plugin for skywalking native.

@vietj, the new tracing features in Vert.x 4.0 look easy and simple to use. As far as I understand though, there isn't a simple way to do it in Vert.x 3.x and that's the version we are adding native SkyWalking tracing support to.
SkyWalking already supports OpenTracing it sounds like Vert.x 4.0 will be automatically supported without a native plugin, so that's awesome.

@BFergerson I wouldn't expect OpenTracing works very well. Sadly to say so. But we could take advances of vertx core to make auto instrument better and easier.

@wu-sheng, could you explain that please? OpenTracing support is mentioned on this project several times. It's my understanding that if you support OpenTracing then you support anything that uses OpenTracing compatible tracers?
vietj also mentions that they will have Zipkin support for Vert.x 4.0. Even with OpenTracing and Zipkin native support for Vert.x we would still need an auto-instrumentation plugin?

ould you explain that please? OpenTracing support is mentioned on this project several times. It's my understanding that if you support OpenTracing then you support anything that uses OpenTracing compatible tracers?

That is the typical issue of OpenTracing. OpenTracing is just a set of API, it is no guarantee. And for SkyWalking, it is just a bridge API to agent core, which means, sync call most likely works, but requires user manually active SkyWalking tracer, based on OT API, but in the real world, most users would not because we have the agent, people want really auto, right?

vietj also mentions that they will have Zipkin support for Vert.x 4.0. Even with OpenTracing and Zipkin native support for Vert.x we would still need an auto-instrumentation plugin?

If we just want Zipkin trace, and work with other zipkin plugin, then you don't need auto-instrument agent. If you want SkyWalking format and work with other our auto-instrument plugins, then YES, we still need the plugin.

Basically, the auto-instrument magic means, in almost all cases, plugins are required. In SkyWalking most known users, they only use OT-SkyWalking API to set some tag and local span.

I mostly work around auto-instrument, especially in Java. @adriancole knows much more about the OpenTracing issues in manual instrumentation library compatible. It is far from your expected right now.

Two more things.

OpenTracing Java 0.32 breaks 0.31 compatible again, and our community has no one wants to do upgrade right now. Sadly.
OpenTracing and OpenCensus are going to be merged, so API will most likely be broken again.

You see, there are a lot of things happening, and I and most SkyWalking users/contributors want 100% stable and reliable, we recommend our native format and plugins always. We are tracing and monitoring a lot of systems, we can't break them.

@wu-sheng you can try to ignore the opentracing thing and look at the other one in vert.x 4. I think @vietj did bake in some OT semantics.. some are easy to fix. however OT and brave apis are similar anyway. it might be more difficult to take something not largely a clone of brave and integrate it https://github.com/vietj/vertx-tracing/pull/1/files#r274201967

We can close this now with https://github.com/apache/incubator-skywalking/pull/2386 being accepted. This gives Skywalking basic support for Vert.x. As I continue to monitor my Vert.x applications with Skywalking I'll expand on this support. Thanks for all the help @wu-sheng. Looking forward to the release of 6.1 ;).

@BFergerson Thanks. Welcome to contribute other plugins. The 6.1 release should happen next month.


Feature/visualisation

Coverage decreased (-8.5%) to 86.815% when pulling 6f9a1efbe49d309f4d72b273b57b63777b37dc09 on itsvipa:feature/visualisation into 9d43c8d10e5cfa6870d5b89c9ae779f05802cd8d on xethorn:master.


Sector Page Update

https://www.vatsim.uk/operations/sectors
Top down order for TC SE needs to include LON_D_CTR before LON_S_CTR.
Relevant to:
London/Heathrow (EGLL), London/City (EGLC), Southend (EGMC), Biggin Hill (EGKB), Lydd (EGMD)

Also Teeside should be Teesside


Possibly change formatters

What do you all think about updating this to use black instead of yapf?
https://github.com/psf/black
It seems more like prettier for javascript which we have really come to like.
Would this be a cheese move or do you see some value in using a more active/popular/maybe faster package?

I absolutely love prettier so if this is a step closer to that standard, I'm all for it. It does seem like it's much more active than yapf. Also, switching formatters isn't a huge deal. I'd definitely be up for trying it out.


Modular Receiver Is Invisible

It appears as an error in menus, and invisible when I saw it as maints loot. Not much else I can add.

fixed


Find duplicate reaches

pyGNAT needs to be able to find duplicate stream reaches in a network (which can happen occasionally).  Not sure if this could be done with NetworkX or PyQGIS.

So networkX filters out duplicates unless they are flowing in opposite directions.
I think we'll probably need to use Shapely for this. Can we loop over all line segments and see if their start and end points match any other line segments?
then you could just use a python list trick to find the duplicates
Something like this (sorry, it's just pseudocode)
from collections import Counter
# get all the start and end points in a big list
endbits = [(line.start, line.end) for line in shapefile]
# find any duplicates
duplicates = [k for k,v in Counter(endbits).items() if v>1]

I'm thinking that since duplicates are filtered out, this kind of solves our problem.  Why do we need to identify them?

@jesselangdon We can talk about this when I come down but there is a good case for it when considering braids. What you consider a duplicate reach might disagree with what networkx thinks it is.
I've started using MultiDiGraphs for everything for this very reason. It adds a little extra complexity but I think there's a payoff in the end.

See commit https://github.com/SouthForkResearch/pyGNAT/commit/a245880a3ce0710b9fec22255ae1c6626e725a08


stateful LSTM can't learn periodicity with batch_size=1

After familiarizing myself with keras' LSTM examples and tutorials (e.g. https://github.com/fchollet/keras/blob/master/examples/stateful_lstm.py), and discussions on the "stateful" flag (e.g. http://philipperemy.github.io/keras-stateful-lstm/, http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/) , I modified the keras/stateful_lstm.py example so that the network is trained to learn an 'echo' (or delay) instead of fitting a decaying cosine.
The keras stateful_lstm.py example uses batch_size=25.  Other examples & tutorials seem to prefer using batch_size=1 for stateful LSTM networks.   I find that for batch_size=1, such a network does not learn the periodicity (echo) requested.  What happens with batch_size=1 is simply that the output mirrors the input, with no echo.   And with batch_size=25, only delays which are integer multiples of 25 can be learned.  I'd like to be able to use batch_size=1 so that in principle any delay can be learned.
It seems only way the network seems to learn is if batch_size(>1) is a factor of the delay length in samples (variable "delay_num_samples" in gist), i.e. only if the delay length is an integer multiple of batch_size.  Since delay length wouldn't be known a priori, this is a problem.
There doesn't seem to be a clear reason why batch_size=1 shouldn't work.  We are simply evaluating the loss function every time instead of averaging over batches of 25 or 50, and still going through the whole dataset each epoch.
Link to code gist, modeled after examples:
https://gist.github.com/drscotthawley/cd4b21c110eefd53c553dc6283843094
Checklist:

Latest version of keras
Tried both (latest) Tensorflow and Theano backends.
Had posted to keras_users, long time no response, discovered batch_size sensitivity (i.e. that it can work for certain batch sizes but not for 1) and have escalated here to Issues.

More directly, if one simply sets batch_size=1 in the original https://github.com/fchollet/keras/blob/master/examples/stateful_lstm.py, then the result is that the model does not learn the decaying cosine.

This is not a bug.  Just an expression of why mini-batches are often better at optimizing.
Closing.

Wouldn't mind hearing more about what you learned. In the U-net paper (classifying pixels) the authors recommended a batch size of 1 but a high momentum to carry information across examples. I'd imagine that smooths out the noise as well.


[AIRFLOW-926] Fix JDBC Hook

Dear Airflow maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[X] My PR addresses the following Airflow-926 issues and references them in the PR title.

Description

[X] JayDeBeAPI made a backwards incompatible api change, this change updates airflow's api call.

Tests

[X] My PR adds the following unit tests OR does not need testing for this extremely good reason: Small API Change with no existing tests to update.

Commits

[X] My commits all reference JIRA issues in their subject lines

Note: I've removed several unrelated changes from the original pull request in the hopes that this would gain some traction.

Codecov Report

Merging #2651 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #2651      +/-   ##
==========================================
- Coverage   71.75%   71.75%   -0.01%
==========================================
  Files         152      152
  Lines       11749    11749
==========================================
- Hits         8431     8430       -1
- Misses       3318     3319       +1

Impacted Files
Coverage Δ

airflow/hooks/jdbc_hook.py
0% <0%> (ø)
:arrow_up:

airflow/models.py
87.1% <0%> (-0.05%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f0798f...469a1e7. Read the comment docs.

Can you please update your commit message to refer to what you are fixing and preferably at a test as well?

I've added a more detailed description to the commit.
As far as a test goes I'm not sure how to write a test for this for the following 2 reasons:

External Database? (glancing at 1 of the 2 existing hook tests it looks like mock is the answer but i'm not familiar with it...)
Needs a jar file? (No idea how to solve this without adding a jar file to the airflow package which seems wrong)

if theres no jar file both implementations fail with the same error message Class org.postgresql.Driver not found

However, here is a test script that demonstrates airflow's current issue along with setup & teardown instructions (Requires macos)
import jaydebeapi
import os

## Requirements
# brew cask install postgres
# download postgres jdbc driver to downloads folder
# https://jdbc.postgresql.org/download/postgresql-42.1.4.jar

jdbc_driver_name = 'org.postgresql.Driver'
login = os.getlogin()
host = 'jdbc:postgresql://localhost:5432/' + os.getlogin()
psw = ''
jdbc_driver_loc = '/Users/' + os.getlogin() + '/Downloads/postgresql-42.1.4.jar'

try:  # This is the new JDBC Hook code
    good_conn = jaydebeapi.connect(jclassname=jdbc_driver_name,
                                   url=str(host),
                                   driver_args=[str(login), str(psw)],
                                   jars=jdbc_driver_loc.split(sep=","))
    print('Good succeeded')
except Exception as e:
    print('Good failed with error', e)

try:  # This is the old JDBC Hook code
    bad_conn = jaydebeapi.connect(jdbc_driver_name,
                                  [str(host), str(login), str(psw)],
                                  jdbc_driver_loc,)
    print('bad succeeded')
except Exception as e:
    # fails with
    # No matching overloads found. at native/common/jp_method.cpp:117
    print('bad failed with error', e)

## Cleanup
# brew cask uninstall postgres
# brew cask zap postgres
# rm ~/downloads/postgresql-42.1.4.jar

Thanks for the detailed explanation. Why don't you use mock to see if you are truly calling the connect api with the right parameters? It is just a unit test then, but that is fine.

Thanks @bolkedebruin !


pushState - state object support

Hey, thanks for the awesome library :)
Do you have plans to implement support for the state object param of History.pushState()?
My current use case is to save scroll position between pages. The site I'm working on is fully AJAXed so I can persist a variable with the scroll position between page transitions. But, if a user reloads the page, the previous scroll position is lost. So then if they hit the back button, the page scrolls back to the top regardless of where they were previously.
Seems like the cleanest way to do it would be to just push a state object along with the route to router.navigate(path = '', state = {}, absolute = false). I'm happy to open a PR if you like this idea, or defer to you if you already have this feature in the works.

Hey,
I never hit that use case. Happy to accept a PR fpr such feature though.

Hey @estrattonbailey,
I'm trying to clean up old issues. So, is there any development around your idea. For me it seems like the desired behavior, i.e. keeping the state, should be done outside of the router. What's your motivation about sending it to Navigo. Is it that you want to receive the state in your handler?

Hey @krasimir, apologies on never coming back to close this, thanks for your follow up.
I worked with a forked Navigo with state previously, but in hindsight it makes more sense to just use history.replaceState() to track stuff like scroll position. The little library I was working on actually uses that method now, and works pretty well.
Thanks for you lib! Great stuff 👌


SS-7586 SS-7550 viewer major release update

https://shapediver.atlassian.net/browse/SS-7586
https://shapediver.atlassian.net/browse/SS-7550

@MajorMeerkatThe3rd please check out this branch, after updating to the major viewer release the viewport stays white. Did I miss something?

@MajorMeerkatThe3rd please check out this branch, after updating to the major viewer release the viewport stays white. Did I miss something?

@snabela I already found the issue. This seems to be specific to the major version and react. (I log the properties in the createViewport function now, but with react there is some circularity in the canvas element)
Something on the AppBuilder side: There is actually an error happening and thrown, but it is never displayed somewhere. Please check the code here: https://github.com/shapediver/AppBuilderSdk/blob/cb694ee37e940843ed8c26d1c4d8ecb11e11916f/src/store/useShapeDiverStoreViewer.ts#L54

@MajorMeerkatThe3rd please check out this branch, after updating to the major viewer release the viewport stays white. Did I miss something?

@snabela I already found the issue. This seems to be specific to the major version and react. (I log the properties in the createViewport function now, but with react there is some circularity in the canvas element)
Something on the AppBuilder side: There is actually an error happening and thrown, but it is never displayed somewhere. Please check the code here:
https://github.com/shapediver/AppBuilderSdk/blob/cb694ee37e940843ed8c26d1c4d8ecb11e11916f/src/store/useShapeDiverStoreViewer.ts#L54

Many thanks for the hint! I will improve this as part of this PR!

@MajorMeerkatThe3rd please check out this branch, after updating to the major viewer release the viewport stays white. Did I miss something?

@snabela I already found the issue. This seems to be specific to the major version and react. (I log the properties in the createViewport function now, but with react there is some circularity in the canvas element)
Something on the AppBuilder side: There is actually an error happening and thrown, but it is never displayed somewhere. Please check the code here:
https://github.com/shapediver/AppBuilderSdk/blob/cb694ee37e940843ed8c26d1c4d8ecb11e11916f/src/store/useShapeDiverStoreViewer.ts#L54

Many thanks for the hint! I will improve this as part of this PR!

works now!

@MajorMeerkatThe3rd please check out this branch, after updating to the major viewer release the viewport stays white. Did I miss something?

@snabela I already found the issue. This seems to be specific to the major version and react. (I log the properties in the createViewport function now, but with react there is some circularity in the canvas element)
Something on the AppBuilder side: There is actually an error happening and thrown, but it is never displayed somewhere. Please check the code here:
https://github.com/shapediver/AppBuilderSdk/blob/cb694ee37e940843ed8c26d1c4d8ecb11e11916f/src/store/useShapeDiverStoreViewer.ts#L54

Many thanks for the hint! I will improve this as part of this PR!

works now!

@snabela there was also another issues, should all be resolved now. please try it out

@MajorMeerkatThe3rd please check out this branch, after updating to the major viewer release the viewport stays white. Did I miss something?

@snabela I already found the issue. This seems to be specific to the major version and react. (I log the properties in the createViewport function now, but with react there is some circularity in the canvas element)
Something on the AppBuilder side: There is actually an error happening and thrown, but it is never displayed somewhere. Please check the code here:
https://github.com/shapediver/AppBuilderSdk/blob/cb694ee37e940843ed8c26d1c4d8ecb11e11916f/src/store/useShapeDiverStoreViewer.ts#L54

Many thanks for the hint! I will improve this as part of this PR!

works now!

@snabela there was also another issues, should all be resolved now. please try it out

@MajorMeerkatThe3rd works great now, many thanks! Please let me know once the viewer package is available publicly.

@MajorMeerkatThe3rd short question: do you see a better way to do this, and avoid the casting? https://github.com/shapediver/AppBuilderSdk/blob/53091bf1d0ff5e31d1c58a94df604b7c0b84a98b/src/store/useShapeDiverStoreParameters.ts#L99


fix(*): add readiness & liveness probes

Fixes #92
Fixes #89
Ref deis/deis#4809
When this is done, merge deis/charts#93
Still TODO:

[x] Check k8s server on probe request
[x] Check that the server is actually running on probe request
[x] Tests for healthZHandler
[x] TestOpenCloseConcurrent
[x] make test output: pkg/sshd/server_test.go:106: not enough arguments in call to Serve

Is this still WIP?  The WIP in the title and the "awaiting review" tag are giving me mixed signals.

@krancour sorry about that. Yes, still in progress. I changed the label to indicate so.

Tested myself. To test yourself, spin up a deis-dev cluster and kd delete rc deis-builder && kd create -f manifests/deis-builder-rc.yaml && kd get pod -w. Once you see the pod come up, wait for it to go into Running and `1/1

@krancour this is now ready for review

why not modify fetcher to health serv. Do we need another http server to handle health serv. Already fetcher has a health check handler

Anyways we are thinking of moving away from fetcher . So please Ignore my previous comment

If k8s API is not accessible how come it's builder fault . Also it not making sense to me to check if API is accessible or not as builder wont start in the first place.
I agree checking S3 storage is accessible or not
This is overall amazing code. I feel its more complex design
If we are coming up a different health server why do we need an entire class to atomically set a value. We can just write one  more handler in the health server that sets the circuit state value and ssh.server.go calls that handler before this https://github.com/deis/builder/blob/master/pkg/sshd/server.go#L73 and health check handle polls change of circuit state till the timeout is over.

Also liveness probes shouldn't be checking S3 API accessibility

Already server handles ping request, but ping method wasn't implemented https://github.com/deis/builder/blob/master/pkg/sshd/server.go#L202. I think liveness probe should call ping

If k8s API is not accessible how come it's builder fault

It's not the builder's fault, but the builder shouldn't be able to accept any request to do work when object storage, a dependency without which it can't do its job, is unavailable to it.

If we are coming up a different health server why do we need an entire class to atomically set a value

We don't necessarily need this Circuit type, but it presents the simplest abstraction between the SSH server and the health check server I could find. The basic requirement is that the SSH server can communicate its status to the health server in a concurrency safe manner.
I tried a few other implementations, including a sync.WaitGroup implementation and a channel based implementation, but both ended up being more code with more complexity.

We can just write one more handler in the health server that sets the circuit state value and ssh.server.go calls that handler before this https://github.com/deis/builder/blob/master/pkg/sshd/server.go#L73

I'm not 100% sure I understand what you mean here, but the closure of the circuit happens right above that line here, and that is the logic that effectively communicates to the health check server that the ssh server is ready
IMO It doesn't make as much sense to call an HTTP endpoint than to make a function call from within the same process.

health check handle polls change of circuit state till the timeout is over.

The health check server essentially does poll the circuit state because Kubernetes polls the health check  endpoint. I opted not to implement any polling logic in the health check endpoint in favor of just reusing the Kubernetes logic.

Already server handles ping request, but ping method wasn't implemented https://github.com/deis/builder/blob/master/pkg/sshd/server.go#L202. I think liveness probe should call ping

I think implementing the health check logic in the SSH ping handler is suboptimal. Here's my thinking:

The readiness and liveness probes could become expensive in the future as the SSH server grows in complexity. In fact, they may add more CPU load onto the existing server (I have no data on this, purely a guess)
The health check logic would be unnecessarily mixed with the SSH logic, violating separation of concerns. This point also relates to the previous
The end-user expects to do an ssh ping operation to determine whether the server up and ready for their git pushes. By putting health check logic as-is into the ping (and I believe the health check logic we have now is 100% necessary), we allow any end-user of the builder to (possibly unwillingly or unknowingly) exercise our object store and the Kubernetes API arbitrarily.

@bacongobbler are you referring to the AbsPath("/healthz") bug?

Sorry, that comment was in regards to refactoring out pkg/log to use the standard log package as in this commit. Github doesn't really show well that I was making a comment on that commit.

Ah, got it. I see the tiny text where it says you commented on the commit now.
Filed https://github.com/deis/pkg/issues/19 to track

Code LGTM

Gonna hold off on merging this until the discussion at https://github.com/deis/builder/pull/149#issuecomment-183126063 is complete

FWIW, here is my view on health checking logic:

The readiness check should ONLY pass if the underlying system believes itself to be ready to successfully handle a request.
A liveness probe should ONLY pass if the system is in a condition where it could successfully handle a request.

Neither probes are about establishing "fault", the purpose of the probes is to indicate that the system believes itself to be in a position to fulfill its service contract. That's it. If it can't reach a database, cache, API endpoint, etc., it should fail its check.
That said, I am a little concerned with probes being dependent on the state of Kubernetes itself. So the k8s API service availability is sort of a gray area. Originally, I was opposed to failing healthz based on k8s API server problems, but I've changed my mind. Here's the use case that caused me to change my mind:

Builder is working just fine.
A well-meaning admin tries to lock down the API server by enforcing stricter security policies
Builder can no longer hit the API server.

In this case, builder SHOULD fail because it can no longer fulfill its service contract. If a policy was changed, k8s itself will restart the pod and it will come up successful. Otherwise, it will go into crash backoff, which is exactly what ought to happen.

A liveness probe should not fail if k8s is down or s3 is reachable and here is why:
A liveness probe will cause a pod to restart when those external components fail, which in turn means the user can not even do git push but instead gets a unreachable hostname if there are no builders to service the request.
As a user (developer) I want to do a git push which fails gracefully if dependant services are down with a good error message, along the lines of  "unable to process the git push right now, X and Y are down".
Builder is alive and can give good information back to the user, making the liveness probe depend on 3rd party things is a bad UX since the whole builder becomes unavailable at that point. I'm proposing we do the deeper health check during git push and readiness probe but not liveness probe. Different purposes and different external informational needs by devs / operators.
I'd like to hear the opposite side on how we'd handle the scenario of builder being down and telling the developers (and operators) what is going on and how they would effectively debug things when the builder is effectively left in a "running but not ready" state until s3 / k8s / etc comes up?

To chime in my thoughts as well:

In this case, builder SHOULD fail because it can no longer fulfill its service contract. If a policy was changed, k8s itself will restart the pod and it will come up successful. Otherwise, it will go into crash backoff, which is exactly what ought to happen.

This is great for an administrator, but not for a user. In the current implementation the user has no view into why the builder is down. If we were to go down this route, a status page (similar to status pages like http://status.docker.com/) would be invaluable to a user to see what systems are down, bridging the communication gap between users and administrators.

I'd like to hear the opposite side on how we'd handle the scenario of builder being down and telling the developers (and operators) what is going on and how they would effectively debug things when the builder is effectively left in a "running but not ready" state until s3 / k8s / etc comes up?

^^
Another proposed solution offline was to gracefully fail a git push, but internally perform health checks on upstream services like k8s/minio, which would then be logged if they're reporting as down. That way we can pick that up in https://github.com/deis/monitor with alerting (not implemented in telegraf AFAIK but not something that we couldn't do ourselves)

Ok, after extensive internal discussion, consensus is as follows:

For now, and probably through the v2-beta release, the logic in this PR will suffice as-is
I will be creating issues for the following enhancements, to be done at various points in the release cycle

Have health probes check the status of the controller
Have health probes ping the local SSH server (instead of what they do now, which is only verify that it has started properly)
Have the git push logic respond with an informative error message during downstream dependency failures


use debox instead of scala Map (see benchmark)

https://github.com/non/debox

does not bring much. Should investigate how interating through the result can be improved as most of the time is spent there


springmvc整合阿波罗时，集群的配置为何不能在app.properties中完成而需要在server.xml中完成，很不方便

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Hi @chelseXie , seems like your question is not about our project, so we will close the issue for now. Maybe you can ask for some help in https://github.com/ctripcorp/apollo/issues. If you have any additional question about the apollo autonomous driving system, please feel free to open a new issue.
Thank you for supporting Apollo!


Remove distutils

Pull Request Description
What issue does this change request address?
What are the significant changes in functionality due to this change request?
Removes use of distutils since it is removed in Python 3.12

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[ ] 1. Review all computer code.
[ ] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[ ] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
[ ] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True.
[ ] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[ ] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[ ] 8. If an analytic test is changed/added is the the analytic documentation updated/added?
[ ] 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

Job Test qsubs sawtooth on b9f899a : invalidated by @joshua-cogliati-inl
failed in fetch

Job Test qsubs sawtooth on b9f899a : invalidated by @joshua-cogliati-inl
restarted civet

Job Test qsubs sawtooth on b9f899a : invalidated by @joshua-cogliati-inl
failed in set python environment

Job Test qsubs sawtooth on b9f899a : invalidated by @joshua-cogliati-inl
Diff tests/cluster_tests/test_mpiqsub_long Diff tests/cluster_tests/test_pbsdsh

Job Test qsubs sawtooth on b9f899a : invalidated by @joshua-cogliati-inl
RAVEN was updated.

Tests passed, and checklist is good.


Feature/advanced indexing

A new test that:

creates a tool
refreshes the tool
perform a full re-index
add labels to the tool

a specified number of times.  Eventually, the elastic manager should be able to see which tools have changed and do partial re-index instead of doing a full one every time.
This will fail on Travis until the swagger fix is in.

note re-run tests after #843 is squashed/rebased

To be clear, this looks like this adds a test and a future foundation for advanced indexing. But actual implementation is in a future PR

Yes, it only adds a test right now.

As discussed, this doesn't really need to be a test. This can just be a program that we run to benchmark our use of elastic search. (Add an entry in Dockstore primer or the readme.md)

Coverage decreased (-0.1%) to 72.243% when pulling 0cc79338e389eebe30d5e489153e321329ea83ac on feature/advancedIndexing into 8e23191080afaa3515c7ba4c774a11004ed1711f on develop.


Build / Deploy Documentation

A big part of this is already done on the Readmes that you can view on github!
As I wrote the Build Instructions for the Backend, it would be interesting, if someone else reviews them and gives feedback to me on what could be improved. Maybe @s-kruschel ?
There still have to be instructions written on how to build the frontend (install yarn with npm, and install packages with yarn... I think with expo you have the possibility to build apks, someone should make detailed instructions on that).


[acr] update sdk version / requirements.*.txt

Description of PR (Mandatory)
(Why this PR? What is changed? What is the effect? etc. A high-quality description can accelerate the review process)
Testing Guide
(Example commands with explanations)
History Notes:
(Fill in the following template if multiple notes are needed, otherwise PR title will be used for history note.)
[Component Name 1] (BREAKING CHANGE:) (az command:) make some customer-facing change.
[Component Name 2] (BREAKING CHANGE:) (az command:) make some customer-facing change.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

@adewaleo do you want a hotfix release? If so, please submit a PR to merge into release branch.

Fixes #12819

I will submit a new PR to merge into release branch.

can you please tag me to follow up @fengzhou-msft ?
Thank you.


Revert 0.19.0

what

Reverts the updates from #53 as they were no good.

why

Using the flag to not write the back config file causes terraform to believe the file should be destroyed since it was no longer tracked as a resource. That approach just generally won't work.

references

Reverts #53

/test all