GH-37753: [C++][Gandiva] Add external function registry support

Add external function registry support to Gandiva. JSON based function registry can be used for describing the function metadata, and LLVM bitcode can be automatically loaded as pre-compiled external functions.
Rationale for this change
This PR tries to enhance Gandiva by supporting external function registry, so that developers can author third party functions without modifying Gandiva's core codebase. See https://github.com/apache/arrow/issues/37753 for more details.
What changes are included in this PR?
This PR implements basic support of external function registry. It primarily implements two parts to make it work:

define a JSON based function registry format to describe the function metadata, and support loading and parsing such JSON based function registry files from some directory
automatically load and parse LLVM bitcode from files, so that third party pre-compiled functions can be integrated via LLVM bitcode

The overall flow looks like this:

Are these changes tested?
Some unit tests are added to verify this enhancement
Are there any user-facing changes?
No change to the existing behavior. But some new ways to interfacing the library are added in this PR.

Closes: #37753

Notes

Performance

since the function registry is loaded once and stored as a static map internally, there shouldn't too much performance impact typically by externally loading the JSON function registry files.
If external function bitcodes are added, loading them from disk may take some additional time (but it won't affect anything if no external function bitcode are added). I am not sure if we should cache such parsed bitcode in memory for better performance, please shed some light on this. Thanks so much.

Integration with other programming languages via LLVM IR/bitcode

So far I only added an external C++ function in the codebase for unit testing purpose. Rust based function is possible but I gave it a try and found another issue (Rust has std lib which needs to be processed in different approach), I will do some exploration for other languages such as zig later.

Non pre-compiled functions, may require some different approach to get the function pointer, and we may discuss and work on it in a separated PR later. I am currently thinking using approach like loading shared libraries during runtime, and find the corresponding function pointer symbols from the shared library, but I don't do any experiment yet so I am not sure if it works this way. Any comment on how this should be done is appreciated.
Certain complex data types, such as map, struct, are not allowed to be used in registry yet, as far as I know, Gandiva doesn't support using these types as input/output for function either, but the registry format could be extended to support them in the future if needed
JSON schema for the external registry format

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "functions": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string"
          },
          "aliases": {
            "type": "array",
            "items": {
              "type": "string"
            }
          },
          "param_types": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "type": {
                  "type": "string",
                  "enum": ["null", "boolean", "uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "float16", "float32", "float64", "utf8", "large_utf8", "binary", "large_binary", "date32", "date64", "day_time_interval", "month_interval", "timestamp", "decimal", "decimal128", "decimal256", "list"]
                },
                "unit": {
                  "type": "string",
                  "enum": ["second", "milli", "micro", "nano"]
                },
                "precision": {
                  "type": "integer"
                },
                "scale": {
                  "type": "integer"
                },
                "value_type": {
                  "$ref": "#/properties/functions/items/properties/param_types/items"
                }
              },
              "required": ["type"]
            }
          },
          "return_type": {
            "$ref": "#/properties/functions/items/properties/param_types/items"
          },
          "result_nullable": {
            "type": "string",
            "enum": ["ifnull", "never", "internal"]
          },
          "pc_name": {
            "type": "string"
          },
          "needs_context": {
            "type": "boolean"
          },
          "needs_function_holder": {
            "type": "boolean"
          },
          "can_return_errors": {
            "type": "boolean"
          }
        },
        "required": ["name"]
      }
    }
  },
  "required": ["functions"]
}

(Please mention me when this is ready for review.)

@kou this PR is ready for review now. Thanks for the help.
There are still 4 CI checks failing. But they seemed to be a failure about test_python_version, which is probably irrelevant with this PR.

Could you rebase on the Dev / Source Release and Merge Script failures?
Could you check the failure on AppVeyor?
https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48091110#L2859
[ RUN      ] TestFunctionRegistry.LookupMultipleFuncs
C:/projects/arrow/cpp/src/gandiva/function_registry_test.cc(105): error: Expected: (say_hello_func) != (nullptr), actual: NULL vs (nullptr)
unknown file: error: SEH exception with code 0xc0000005 thrown in the test body.
Stack trace:
[  FAILED  ] TestFunctionRegistry.LookupMultipleFuncs (9 ms)
[ RUN      ] TestFunctionRegistry.LookupExternalFuncs
C:/projects/arrow/cpp/src/gandiva/function_registry_test.cc(125): error: Expected: (func) != (nullptr), actual: NULL vs (nullptr)
unknown file: error: SEH exception with code 0xc0000005 thrown in the test body.
Stack trace:
[  FAILED  ] TestFunctionRegistry.LookupExternalFuncs (12 ms)
[----------] 5 tests from TestFunctionRegistry (41 ms total)
[----------] 6 tests from TestExternalFunctionRegistry

@kou I've rebased to the latest commit (772a01c08), but the CI ran into another issue (https://github.com/apache/arrow/actions/runs/6290902281/job/17078761957?pr=37787), which seems to be related with LLVM 17 (https://github.com/apache/arrow/issues/37834)? Do I have to fix that or is there any specific commit I should rebase onto?

Could you check the failure on AppVeyor?
https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48091110#L2859

Do you know what the AppVeyor check in CI is? Because I cannot reproduce this issue locally (A M1 MacBook), and several CI checks passed as well. I would like to know what the AppVeyor check in CI is and how it is different compared to other C++ checks in CI to see if I can figure out why they failed in that environment.

@kou I've rebased to the latest commit (772a01c), but the CI ran into another issue (https://github.com/apache/arrow/actions/runs/6290902281/job/17078761957?pr=37787), which seems to be related with LLVM 17 (#37834)? Do I have to fix that or is there any specific commit I should rebase onto?

Oh. We can ignore the failures. We can work on it in a separated PR.

Could you check the failure on AppVeyor?
https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48091110#L2859

Do you know what the AppVeyor check in CI is? Because I cannot reproduce this issue locally (A M1 MacBook), and several CI checks passed as well. I would like to know what the AppVeyor check in CI is and how it is different compared to other C++ checks in CI to see if I can figure out why they failed in that environment. Thanks

AppVeyor uses Windows and Visual C++. The failure may be related to Windows and/or Visual C++.

AppVeyor uses Windows and Visual C++. The failure may be related to Windows and/or Visual C++.

Previously, I made the PR to pass several Windows related checks, such as "C++ / AMD64 Windows MinGW CLANG64 C++ " and "C++ / AMD64 Windows MinGW MINGW64 C++ " (but now they are failing again due to the LLVM 17 issue). Do you know what are the difference between them and the AppVeyor CI check? Considering both of them are Windows environments, do they differ in compiler primarily (clang vs. Visual C++ compiler)?

Could you rebase on main after #37850 is merged and format *.cmake files by archery lint --cmake-format --fix? See also: https://arrow.apache.org/docs/dev/developers/cpp/development.html#code-style-linting-and-ci

Sure. I will wait for the #37850 to be merged.

Could you add the JSON schema too?
Could you add document for this feature? Or we can defer it to a follow-up PR by creating a new issue for it.
Do you have any recommended folder(s) to add:

JSON schema
the corresponding doc

There are some existing folders, but none of them seem to be a good fit for either the JSON schema or doc (I don't find any doc related directory under the src/gandiva directory)

Could you send an e-mail to<EMAIL_ADDRESS><EMAIL_ADDRESS>https://arrow.apache.org/community/#mailing-lists with [DISCUSS][Gandiva] ... subject to get more attention for this specification?
Sure. I sent an email to the dev mailing list a while ago, and I will collect some feedback and see how I should proceed.

Do you know what are the difference between them and the AppVeyor CI check?

They use GCC/Clang (built on MSYS2 that is an UNIX emulation layer on Windows).
AppVeyor use Visual C++.

Considering both of them are Windows environments, do they differ in compiler primarily (clang vs. Visual C++ compiler)?

Yes.

Do you have any recommended folder(s) to add:

JSON schema

How about https://github.com/apache/arrow/tree/main/format (format/gandiva/)?

the corresponding doc

How about https://github.com/apache/arrow/tree/main/docs/source/format (docs/source/format/Gandiva.rst)?

Sure. I sent an email to the dev mailing list a while ago, and I will collect some feedback and see how I should proceed.

Thanks!
The thread: https://lists.apache.org/thread/lm4sbw61w9cl7fsmo7tz3gvkq0ox6rod

Are you aware that the code still includes a header file that doesn't exist in LLVM-17?

After discussing this proposal in mailing list, I will close this PR in favor of https://github.com/apache/arrow/pull/38116 Thank you all for the help!


Adding url and description to setup.py

Adding url and description to setup.py

Thanks for the contribution.  But it broke the build because it doesn't respect the PEP8.  Would you fix this issue?

Merged by hand.


Change provider without exiting the app

Summary
app working fine n good but i didnt find continue to watch tab.
sflix works smooth n good.
suggestion:
its just a suggestion to choose n change provider without exit from app.
note:
all movies provider home tab not loading, othertabs movies and series working good
request:
implement anime series and movies content to watch with English subtitle. (if possible)

thank u very much for ur service n best quality provider which u chosen....
Please confirm the following

[x] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.

suggestion:
its just a suggestion to choose n change provider without exit from app.

I also think it would be a nice feature to be able to switch providers from within the app, instead of having to leave it.
But this functionality is not a priority at all and I still have to think about how to implement it well.

note:
all movies provider home tab not loading, othertabs movies and series working good

The AllMoviesForYou provider sometimes takes a long time to load, I need to take a look at this issue.

request:
implement anime series and movies content to watch with English subtitle. (if possible)

If the subtitles are not available on the provider, there is nothing I can do to add them.
However, if the subtitles are available on the provider's site but not on my app, could you provide which films or episodes of series are available with subtitles?

sure okay sir. let me check n let u know by today
On Wed, Dec 7, 2022 at 6:15 PM Lory-Stan TANASI @.***>
wrote:

app working fine n good but i didnt find continue to watch tab. sflix
works smooth n good.
It seems to work fine on my device.
Could you follow these steps to make sure this feature works for you:

Watch a random movie
Seek to ~50% of the video
Go back to provider's homepage
Check if your movie shows up in a Continue Watching row
Click on this movie
The app should restart the movie where you left off

https://user-images.githubusercontent.com/46174829/206163341-eeb3fedf-6930-4f48-921d-23ad31693e05.mp4
If it's not working for you, please report a bug here
https://github.com/stantanasi/sflix/issues/new/choose!
—
Reply to this email directly, view it on GitHub
https://github.com/stantanasi/sflix/issues/15#issuecomment-1340915521,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNL4ZF3V37AADGJQLOYDM3WMCBH3ANCNFSM6AAAAAASWMX2VU
.
You are receiving this because you authored the thread.Message ID:
@.***>

Can you add Arabic subtitles please

Available on Release 1.3.0


NTR: abnormal eMTOC assembly

See:
https://github.com/pombase/curation/issues/1076

abnormal equatorial microtubule organizing center assembly FYPO:0005690
edit file: 774d5dd9b89cea0676a37c022a8b71b72d5d1585
release: e0bbc398974d9c8b721de72cddd6079c2c82ce6a


Incorrect return type

The documentation states that:

Name
Type
Required
Description

successCallback
Function
No
A function to call when a record is created. An object with the following properties will be passed to identify the new record:entityType: String. The entity logical name of the new record.id: String. GUID of the new record.

but then it states that return value for the function call is Promise<Entityreference> which is different than the successCallback definition.
The actual call to the function in an online instance (version 9.1.0000.22242) returns the value described in the successCallback.
But the sample component TS_WebAPI and the powerapps-component-framework typings use the Promise<EntityReference>

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1d976623-48b2-06bb-f661-48c96c5f93fd
Version Independent ID: 511dec41-fd89-5007-0467-c7f213fd2ea6
Content: createRecord - Power Apps
Content Source: powerapps-docs/developer/component-framework/reference/webapi/createrecord.md
Service: powerapps
GitHub Login: @Nkrb
Microsoft Alias: nabuthuk

@BetimBeja Thanks for contacting us. We have created a internal work item for the respective dev team to update the return type. Will update the docs when the issue is resolved. Closing the issue.  If you need further help, please reopen the issue or open a new issue.


Update ebucoreplus.owl

Fix for #278
Proper German and French text is needed. @JuergenGrupp @aro-max

Editorial Committee Dec 4 agreed.

minor change , will be process later


feat: support Starknet mainnet

We need to add support for Starknet mainnet in our UI. This will require some change on SX.js, SX-UI, SX-API and Mana most likely. I can help setup the server, we will move SN testnet API to https://testnet.api-1.snapshotx.xyz and use https://api-1.snapshotx.xyz for SN mainnet.

@bonustrack sx.js already has mainnet config so should be good to go. So is mana (just needs to get account set up on mainnet and get funded).
Only sx-api and sx-ui should need changes, I will work on those and we can deploy updated sx-api.

Got it sound great!

This endpoint for testnet works now: https://testnet-api-1.snapshotx.xyz


bintrayUpload gradle task throws 'could not create package' error

Error:Execution failed for task ':bintrayUpload'.

Could not create package 'user_name/pos/sdk': HTTP/1.1 404 Not Found [message:Repo 'pos' was not found]

buildscript {
    repositories {
        jcenter()
    }

    apply plugin: 'java'

    dependencies {
        classpath 'com.android.tools.build:gradle:2.2.3'
    }
}

plugins {
    id 'com.jfrog.bintray' version '1.7.3'
}

bintray {
    user = 'user_name'
    key = 'api_key'
    pkg {
        repo = 'pos'
        name = 'sdk'
        licenses = ['license_name']
        vcsUrl = 'my_url'
        version {
            name = '0.0.1'
        }
    }
    filesSpec {
        from 'sdk/build/outputs/aar/sdk-release.aar'
        into '.'
    }
}

allprojects {
    repositories {
        jcenter()
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

Thanks

@Smekalisty,
I assume there's no pos repository in Bintray for this user, as the Bintray response message states.

Please see the user name 'donov'
Thanks

Maybe it helps somebody; from plugin Getting Started:

userOrg = 'bintray_user'

Need to fill solely organisation name, and not bintray user name, and for me this field is required, not optional
Thanks Eyal @eyalbe4 for help by email


Option to run dataloader on single process for distributed training

What is your question?
Is there a way to run dataloading on a single process for DDP distributed training?  As is, pytorch-lightning creates a different dataloader on each process.  While this is fine with a normal dataloader that uses a distributed sampler, I am using an IterableDataset and my batches are duplicated on every GPU, rendering multi-GPU training useless.  I would like to be able to sequentially submit batches to each GPU so that they can all simply use batches from the same IterableDataset.  Documentation from DistributedDataParallel indicates that this is supported behavior
What have you tried?
I have tried all of the different data utilities offered in pytorch-lightning, including the new LightningDataModule in the latest release.  All of them seem to duplicate the dataloader on each GPU, leading to duplicate batches.
What's your environment?

OS: Linux
Packaging: pip
Version: 0.9.0rc2

@davidkartchner Actually, I'm not sure this is possible. At least I'm not aware of any possibility to do this.
What we're currently doing at lightning is to add a distributed sampler, so that the dataset may be the same, but the set of possible indices for the sampler in each process is different to the other processes, which should also cause unique batches.

@justusschock Thanks for the reply.  I had hoped that this would work in my case, but the distributed sampler doesn't work with an IterableDataset, as noted in the Pytorch-Lightning docs on Multi-GPU training.  This because instances of IterableDataset don't allow random access, but rather return an iterator through the data.  The main use case for iterable datasets is reading in streaming data, i.e. reading line by line from a file, where random index access isn't available.  Performing distributed sampling with IterableDatasets is currently an open issue of development in pytorch (see #28743)
In my case, this is necessary because my text corpus is too large to load into memory, particularly if its duplicated 16 times.  Adding this functionality would be very beneficial for NLP applications, especially related to training language models.

I managed to fix this issue by creating an index of the character offsets of each line in my training data file.  It allows me to read any arbitrary line in the file in constant time but still only read a single line at a time to have the memory footprint of an IterableDataset.
If anyone else runs into this problem, the following had helpful guidelines/code:

RandomAccessFileReader package
Stackoverflow post

Closing this for now.


doctor doctor

@wkalt updated to latest go code - thoughts on what we'd like to see done before merging the first version of doctor?


LightSensor Units

Hi,
In the Home app on iOS, the units for light level are lux. I have set the unit to Percentage and the app does not change the unit and remains as lux.
Example code below
func New(baseTopic string, displayName string) *Sensor {
	acc := Sensor{
		Accessory:         accessory.New(accessory.Info{Name: displayName}, accessory.TypeSensor),
		TemperatureSensor: service.NewTemperatureSensor(),
		LightSensor:       service.NewLightSensor(),
		topic:             baseTopic,
	}

	acc.TemperatureSensor.CurrentTemperature.SetValue(0)
	acc.TemperatureSensor.CurrentTemperature.SetMinValue(-40)
	acc.TemperatureSensor.CurrentTemperature.SetMaxValue(100)
	acc.TemperatureSensor.CurrentTemperature.SetStepValue(0.1)
	acc.AddService(acc.TemperatureSensor.Service)

	acc.LightSensor.CurrentAmbientLightLevel.SetValue(0)
	acc.LightSensor.CurrentAmbientLightLevel.SetMinValue(0)
	acc.LightSensor.CurrentAmbientLightLevel.SetMaxValue(100)
	acc.LightSensor.CurrentAmbientLightLevel.SetStepValue(1)
	acc.LightSensor.CurrentAmbientLightLevel.Unit = characteristic.UnitPercentage
	acc.AddService(acc.LightSensor.Service)

	return &acc
}

Does anyone have any ideas, why the unit is not changing in iOS? I have also tried this with a custom light sensor type, with the same result.
Thanks

I suspect that the unit of a predefined characteristic cannot be changed – HomeKit expects lux as the unit for the current ambient light level characteristic.
Why not use a read-only brightness characteristic instead?

Thanks, that sounds like a sensible idea!


Fix some broken markdown.

I noticed that some of the markdown in the project's README was broken on Github. This PR just fixes that.

Thanks for the fixes!


4.3.1 tagged at wrong commit

The 4.3.1 tag was made at this commit - https://github.com/resque/resque-scheduler/commit/dbd290ac77266ad2c91c6f87422492cf660d224a - which updated the internal version number.
However, several PRs were merged and the changelog updated in subsequent commits:

Can we get a 4.3.2 out to actually get these fixes?

Non-issue now that 4.4.0 is released.


refactor: remove toolkit features

Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

Move controlls、frameBufferPicker、Stats to  https://github.com/oasis-engine/engine-toolkit

Codecov Report

Merging #780 (5b7f775) into dev/0.8 (d0e639c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           dev/0.8     #780   +/-   ##
========================================
  Coverage    10.31%   10.31%
========================================
  Files          242      242
  Lines        11427    11427
  Branches      1708     1708
========================================
  Hits          1179     1179
  Misses       10045    10045
  Partials       203      203

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d0e639c...5b7f775. Read the comment docs.


Add Input Method

Allow user input with builtin input method.

commit: https://github.com/johneastman/boomerang/commit/7ec2a48b3ef34614ba5b65233c1833e3a04af601


Updates for Secrets Store CSI Driver on Kubernetes

Ensure that your Kubernetes version is 1.16.0 or later.

We should also mention the need to have 1.18+ Kubernetes version for Windows clusters.

To create your own custom SecretProviderClass object with provider-specific parameters for the Secrets Store CSI driver, use this template. This object will provide identity access to your key vault.

The use this template link is broken. It needs to be updated to https://github.com/Azure/secrets-store-csi-driver-provider-azure/blob/master/examples/v1alpha1_secretproviderclass_service_principal.yaml

userAssignedIdentityID: The client ID of the service principal

This is not right. This is the clientID of the user assigned identity. All the fields are documented here - https://github.com/Azure/secrets-store-csi-driver-provider-azure#create-a-new-azure-key-vault-resource-or-use-an-existing-one

resourceGroup: The name of the resource group
subscriptionId: The subscription ID of your key vault

These are no longer required. They were only required in version < 0.0.4 of Azure Key Vault Provider for Secrets Store CSI Driver.

If you're using managed identities, set the usePodIdentity value as true, and set the userAssignedIdentityID value as a pair of quotation marks ("").

If using managed identities, then pod-identity has to set to be set to false. They are not supported together.

userAssignedIdentityID: "servicePrincipalClientID"       # [REQUIRED] If you're using a service principal, use the client id to specify which user-assigned managed identity to use. If you're using a user-assigned identity as the VM's managed identity, specify the identity's client id. If the value is empty, it defaults to use the system-assigned identity on the VM

This is not right. This field is only set if using user assigned managed identity.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: eee999c1-32dd-d014-40df-8027adee1258
Version Independent ID: 19a29c09-79f8-580a-841f-873e2801da56
Content: Integrate Azure Key Vault with Kubernetes
Content Source: articles/key-vault/general/key-vault-integrate-kubernetes.md
Service: key-vault
GitHub Login: @taytran0
Microsoft Alias: t-trtr

cc @shashishailaj

@aramase Thank you for your insights. I will have this reviewed with the content team and then update the document after that. In case of issues I will reach out to you offline.

Hey @aramase thank you so much for catching these issues. I went ahead and updated the document with your recommendations. The PR has been submitted and the changes will go live in 48-72 hrs. Thanks again! I am proceeding to archive this request for now.

#please-close


[WIP] Django Admin Reskin

Fixes: #

Reskin of Nautobot admin to make it more inline with the rest of the UI.

+2000 lines of HTML is a bit daunting... is this basically a forklift import of django-admin-bootstrapped? If so we want to make sure we give credit where credit is due (it's Apache-2.0 licensed, at least, so no concerns about licensing compatibility).

I thought we were going down the route of converting these to be completely native?

At the moment its +2000 lines, but once the PR is ready I will remove all the unmodified files as they dont need to be overloaded. Kept them in for the moment to make it easier to develop the feature

I thought we were going down the route of converting these to be completely native?

We decided against getting rid of django-admin altogether as it's useful for plugins to be able to make use of it; additionally admin-specific functionality such as user/group/permission management makes sense to not have taking up space in the regular UI as it's not applicable or available to the majority of users.

so presumably everything under extras and periodic tasks will be removed from the admin, if not now, at some point?


Hello,
I got the impression that the development's direction is actually the opposite of the title.
I experiencing a lot of small issues. I caught myself reloading the editor too often lately, hoping this workaround helps.
I think vscode's popularity is based on that it is fast, multi-platform, lightweight, community driven (extensions) and fast (again).
Features are important but I don't think about vscode as a real competitor of MSVS or Xcode. So seeing the tons of features following a lot of issues made me thinking..
As a humble user of vscode, I am kindly asking you to allocate more time on stability please.
Anyway, it is awesome.
Bests
PS.: I know a lot of features (built-in extensions) are in different repos. I wont post this issue every one of it. :)

Thanks for taking the time to write up an issue. Fundamentals (stability, performance, etc.) are very important to us and we prioritize these over features.
However, finding and fixing stability/performance issues is hard without reproducible steps, so if you can isolate a problem that is most helpful.
Oftentimes the best way to start is by disabling all of the extensions and re-enabling them one by one to see if you can isolate the problem.
Here are two docs that explain various ways to narrow down a problem as well:
https://github.com/microsoft/vscode/wiki/Performance-Issues
https://github.com/microsoft/vscode/wiki/Native-Crash-Issues
Thanks!

Okay, thank. I will report one when I am encounter again.
However my suggestion referred to the open issues:
There are 53 issues with label freeze-slow-crash and 822 with label bug.


Fix ref for pull request tests

Check out the pull request ref. Same security considerations apply as #133
Adds an ignore for cypress output files (videos/screenshots) and vscode configuration
Always store test videos, even on failure, since that's when they'll be the most useful

#131

@wrycu Safe to merge. Build continues to fail due to missing secrets.


Merge the same event

From #1

Merge the same stacktrace

Convert stacktrace to base64, and store to HashMap
When next event arrived, check HashMap has it.
If HashMap has it, append event to the same issue.

@qrtt1 我用 MD5 來判斷是否有同樣的 StackTrace，但因為會存在 HashMap 裡面，所以 restart 之後就無法得知原本的內容了 XDD


Kubernetes deployment kubo failed with deploy_k8s script

References : https://github.com/cloudfoundry-incubator/kubo-deployment/issues/215
Running the steps given in https://docs-cfcr.cfapps.io/ for bosh kubernetes deployment. but bosh deployment was successful.
While generate kubo manifest it failed giving below error
Manual try:
root@linbgg219:~/kubo-deployment# ./bin/generate_kubo_manifest /root/kubo-env/kubo/ kubocluster b1ebf8f8-e688-4c12-9d4d-ac5422b4496b
Expected to find a map key 'properties' for path '/instance_groups/name=master/jobs/name=cloud-provider/properties' (found map keys: 'name', 'release')
Exit code 1
last output lines for "./bin/deploy_k8s /root/kubo-env/kubo/ kubocluster public"
Succeeded
Expected to find a map key 'properties' for path '/instance_groups/name=master/jobs/name=cloud-provider/properties' (found map keys: 'name', 'release')
Exit code 1

I have tried with latest release but got below error:
Task 16 | 04:39:53 | Preparing deployment: Preparing deployment (00:00:08)
L Error: Job 'cloud-provider' in instance group 'master' specifies link 'cloud-provider', but the release job does not consume it.
Task 16 | 04:40:01 | Error: Job 'cloud-provider' in instance group 'master' specifies link 'cloud-provider', but the release job does not consume it.
Task 16 Started  Fri Feb  2 04:39:53 UTC 2018
Task 16 Finished Fri Feb  2 04:40:01 UTC 2018
Task 16 Duration 00:00:08
Task 16 error
Updating deployment:
Expected task '16' to succeed but state is 'error'
Exit code 1
my cfcr.yml file:
name: kubocluster
releases:

name: kubo-etcd
version: 8
url: https://github.com/pivotal-cf-experimental/kubo-etcd/releases/download/v8/kubo-etcd.8.tgz
sha1: 267bf72c9f4680b6e1cf542283c653abc74469e2
name: kubo
version: latest
name: docker
version: 30.1.4
url: https://github.com/cloudfoundry-community/docker-boshrelease/releases/download/v30.1.4/docker-30.1.4.tgz
sha1: f79a70452b3ddae52f5ac78dd37ff8d9e6ec19e2
name: bosh-dns
sha1: 9231b7c7d2f6aa85e0cb77fdeef7add54fdb3f1a
url: https://github.com/cloudfoundry/bosh-dns-release/releases/tag/v0.2.0
version: 0.2.0

stemcells:

alias: trusty
os: ubuntu-trusty
version: "3468.21"

addons:

name: bosh-dns
include:
stemcells:

os: ubuntu-trusty
jobs:
name: bosh-dns
release: bosh-dns
properties:
cache:
enabled: true
health:
enabled: true
server:
tls: ((/dns_healthcheck_server_tls))
client:
tls: ((/dns_healthcheck_client_tls))

name: bosh-dns-aliases
jobs:

name: kubo-dns-aliases
release: kubo

instance_groups:

name: master
instances: 1
networks:

name: default
azs: [z1,z2,z3]
jobs:
name: secure-var-vcap
release: kubo
name: flanneld
release: kubo
name: kube-apiserver
release: kubo
properties:
admin-username: admin
admin-password: ((kubo-admin-password))
kubelet-password: ((kubelet-password))
kube-proxy-password: ((kube-proxy-password))
kube-controller-manager-password: ((kube-controller-manager-password))
kube-scheduler-password: ((kube-scheduler-password))
route-sync-password: ((route-sync-password))
backend_port: 8443 # Bosh links hack
port: 8443
authorization-mode: rbac
service-account-public-key: ((service-account-key.public_key))
tls:
kubernetes:
ca: ((tls-kubernetes.ca))
certificate: ((tls-kubernetes.certificate))
private_key: ((tls-kubernetes.private_key))
name: kube-controller-manager
release: kubo
properties:
api-token: ((kube-controller-manager-password))
service-account-private-key: ((service-account-key.private_key))
tls:
kubernetes: ((tls-kubernetes))
name: kube-scheduler
release: kubo
properties:
api-token: ((kube-scheduler-password))
tls:
kubernetes: ((tls-kubernetes))
name: kubernetes-system-specs
release: kubo
properties:
api-token: ((kubelet-password))
admin-username: admin
admin-password: ((kubo-admin-password))
authorization-mode: rbac
tls:
kubernetes: ((tls-kubernetes))
heapster: ((tls-heapster))
influxdb: ((tls-influxdb))
kubernetes-dashboard: ((tls-kubernetes-dashboard))
name: etcd
release: kubo-etcd
properties:
etcd:
dns_health_check_host: <IP_ADDRESS> # this is the default BOSH DNS nameserver
advertise_urls_dns_suffix: etcd.cfcr.internal
require_ssl: true
peer_require_ssl: true
delete_data_dir_on_stop: false
ca_cert: ((tls-etcd-server.ca))
server_cert: ((tls-etcd-server.certificate))
server_key: ((tls-etcd-server.private_key))
client_cert: ((tls-etcd-client.certificate))
client_key: ((tls-etcd-client.private_key))
peer_ca_cert: ((tls-etcd-peer.ca))
peer_cert: ((tls-etcd-peer.certificate))
peer_key: ((tls-etcd-peer.private_key))
stemcell: trusty
vm_type: small
persistent_disk: 5120

name: worker
instances: 1
networks:

name: default
azs: [z1,z2,z3]
jobs:
name: secure-var-vcap
release: kubo
name: flanneld
release: kubo
name: docker
release: docker
properties:
bip: <IP_ADDRESS>/24
flannel: true
iptables: false
ip_masq: false
log_level: error
store_dir: /var/vcap/data
storage_driver: overlay
default_ulimits: [ "nofile=65536" ]
tls_cacert: ((tls-docker.ca))
tls_cert:  ((tls-docker.certificate))
tls_key: ((tls-docker.private_key))
env: {}
name: kubelet
release: kubo
properties:
api-token: ((kubelet-password))
tls:
kubernetes: ((tls-kubernetes))
kubelet: ((tls-kubelet))
name: kube-proxy
release: kubo
properties:
tls:
kubernetes: ((tls-kubernetes))
api-token: ((kube-proxy-password))
stemcell: trusty
vm_type: small-highmem
persistent_disk: 10240

update:
canaries: 1
max_in_flight: 1
canary_watch_time: 10000-300000
update_watch_time: 10000-300000
variables:

name: kubo-admin-password
type: password

name: kubelet-password
type: password

name: kube-proxy-password
type: password

name: kube-controller-manager-password
type: password

name: kube-scheduler-password
type: password

name: route-sync-password
type: password

name: kubo_ca
type: certificate
options:
is_ca: true
common_name: ca

name: tls-kubelet
type: certificate
options:
ca: kubo_ca
organization: "system:nodes"
common_name: kubelet.cfcr.internal
alternative_names: []

name: tls-kubernetes
type: certificate
options:
ca: kubo_ca
organization: "system:masters"
common_name: master.cfcr.internal
alternative_names:

<IP_ADDRESS>
kubernetes
kubernetes.default
kubernetes.default.svc
kubernetes.default.svc.cluster.local
master.cfcr.internal

name: service-account-key
type: rsa

name: tls-docker
type: certificate
options:
ca: kubo_ca
common_name: docker.cfcr.internal

name: tls-etcd-server
type: certificate
options:
ca: kubo_ca
common_name: etcd.cfcr.internal
alternative_names:

"*.etcd.cfcr.internal"

name: tls-etcd-client
type: certificate
options:
ca: kubo_ca
common_name: etcdClient

name: tls-etcd-peer
type: certificate
options:
ca: kubo_ca
common_name: "*.etcd.cfcr.internal"

name: tls-heapster
type: certificate
options:
ca: kubo_ca
common_name: heapster
alternative_names:

"heapster.kube-system.svc.cluster.local"

name: tls-influxdb
type: certificate
options:
ca: kubo_ca
common_name: monitoring-influxdb
alternative_names: []

name: kubernetes-dashboard-ca
type: certificate
options:
is_ca: true
common_name: ca

name: tls-kubernetes-dashboard
type: certificate
options:
ca: kubernetes-dashboard-ca
common_name: kubernetesdashboard.cfcr.internal
alternative_names: []

name: /dns_healthcheck_tls_ca
opsname: global_dns_healthcheck_tls_ca
type: certificate
options:
is_ca: true
common_name: dns-healthcheck-tls-ca

name: /dns_healthcheck_server_tls
opsname: global_dns_healthcheck_server_tls
type: certificate
options:
ca: /dns_healthcheck_tls_ca
common_name: health.bosh-dns
extended_key_usage:

server_auth

name: /dns_healthcheck_client_tls
opsname: global_dns_healthcheck_client_tls
type: certificate
options:
ca: /dns_healthcheck_tls_ca
common_name: health.bosh-dns
extended_key_usage:

client_auth

features:
use_dns_addresses: true

Sounds like you aren't using the latest kubo-release. We recently made a few changes due to the cloud-provider and you need an up to date kubo-release with kubo-deployment.

I am taking both from below links correct me if i am wrong.
https://github.com/cloudfoundry-incubator/kubo-deployment
https://github.com/cloudfoundry-incubator/kubo-release

I just worked few months with bosh, also needs some help with kubo deployment.
Can you please let me know clear steps to deploy kubernetes through bosh with public ip address of my org.

@Velann those links are correct.
Going back to the issue. The problem is that the version of kubo-deployment you deployed is using newer features of kubo-release than the version you uploaded. I don't think I made that clear in my earlier statements.
It looks like you tried to use kubo-release v0.13.0 with the latest version of kubo-deployment. I assume you cloned kubo-deployment which is on master and using the latest unreleased kubo-release changes. You need to checkout the v0.13.0 tag of kubo-deployment which we guarantee works with v0.13.0 of kubo-release.
You can also download the tarball contained in this release: https://github.com/cloudfoundry-incubator/kubo-deployment/releases/tag/v0.13.0. Which has the correct version of kubo-deployment.
I'm a little unclear as to what you want the public ip address to point to. If you are just talking about giving your kubernetes api a public ip then that can be done using iaas routing: https://docs-cfcr.cfapps.io/installing/gcp/routing-gcp/

I mean for kubernetes nodes i am creating i.e.. master / worker i want to assign public ip address. not 172.x.x one. For that can you please help me out

Also the link talks about GCP, https://docs-cfcr.cfapps.io/installing/gcp/routing-gcp/ but i m trying on vsphere

Tried with latest 13 version for kubo-deployment and kubo-release tar inside the kubo-deployment, Got the below error:
Task 12 | 15:44:39 | Preparing deployment: Preparing deployment
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Warning: DNS address not available for the link provider instance: master/f47d15c0-c224-4932-9ba8-4b0d0d499cf2
Task 12 | 15:44:50 | Preparing deployment: Preparing deployment (00:00:11)
Task 12 | 15:44:54 | Error: Unable to render instance groups for deployment. Errors are:

Unable to render jobs for instance group 'master'. Errors are:

Unable to render templates for job 'kubernetes-system-specs'. Errors are:

Error filling in template 'rbac_policy.yml.erb' (line 15: Can't find property '["cloud-provider.type"]')

Task 12 Started  Tue Feb  6 15:44:39 UTC 2018
Task 12 Finished Tue Feb  6 15:44:54 UTC 2018
Task 12 Duration 00:00:15
Task 12 error
Updating deployment:
Expected task '12' to succeed but state is 'error'

Looks like the kubo-deployment/manifests/ops-files/iaas/vsphere/cloud-provider.yml wasn't applied to your manifest for some reason. This ops file should always be applied in our deploy_k8s script as long as the iaas is correctly set to vsphere in your director.yml.

I tried the straight deployment with "bosh deploy -d cfcr manifests/cfcr.yml. Didnt used deploy_k8s this time. Can you please let me know is it possible to make the cloud-provider.yml apply to the manifests.

Hi @Velann it appears you're trying to deploy CFCR in an unsupported manner.  Please try deploying by following the documentation https://docs-cfcr.cfapps.io/installing/vsphere/
There should be no need to use generate_kubo_manifest or to bosh deploy the cfcr.yml directly.  These are handled by the deploy_k8s script.

"I mean for kubernetes nodes i am creating i.e.. master / worker i want to assign public ip address. not 172.x.x one. And the kubelet and all entities should work fine. Please help me to do that.."

Can you please provide more detail about what you are trying to achieve with the public IP address?  What service did you want to expose on the public IP address - is it the kubernetes API or a workload managed by kubernetes?

Yes, I tried with deploy_bosh & deploy_k8s but at the end of kubo deployment below error observed,
Task 34 | 17:20:02 | Preparing deployment: Preparing deployment (00:00:09)
Task 34 | 17:20:14 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 34 | 17:20:14 | Updating instance worker: worker/b8718129-261d-4cbe-b51c-594d95d6c9f9 (0) (canary) (00:05:33)
L Error: 'worker/b8718129-261d-4cbe-b51c-594d95d6c9f9 (0)' is not running after update. Review logs for failed jobs: docker, kubelet
Task 34 | 17:25:47 | Error: 'worker/b8718129-261d-4cbe-b51c-594d95d6c9f9 (0)' is not running after update. Review logs for failed jobs: docker, kubelet
Task 34 Started  Thu Feb  8 17:20:02 UTC 2018
Task 34 Finished Thu Feb  8 17:25:47 UTC 2018
Task 34 Duration 00:05:45
Task 34 error
Updating deployment:
Expected task '34' to succeed but state is 'error'
Exit code 1
While checking the logs in worker below error is seen:
Error starting daemon: Error initializing network controller: Error creating default "bridge" network: failed to allocate gateway (<IP_ADDRESS>): No available addresses on this pool
Error starting daemon: Error initializing network controller: Error creating default "bridge" network: failed to allocate gateway (<IP_ADDRESS>): No available addresses on this pool
Error starting daemon: Error initializing network controller: Error creating default "bridge" network: failed to allocate gateway (<IP_ADDRESS>): No available addresses on this pool
Error starting daemon: Error initializing network controller: Error creating default "bridge" network: failed to allocate gateway (<IP_ADDRESS>): No available addresses on this pool
Because of which docker is not started and kubelet is also not starting since docker is down.
Please let me how to correct it.

Hi @Velann,
From the error message Error starting daemon: Error initializing network controller: Error creating default "bridge" network: failed to allocate gateway (<IP_ADDRESS>): No available addresses on this pool, it looks like you might not have enough allocatable IP addresses in your network.  You should check your network settings.


Unify worker pollers options naming between GoSDK and JavaSDK

Synchronization of similar option names helps to keep guides and docs less cluttered with references between different SDKs.

@bergundy agree. But until we have autoscaling, let's just align the names to reduce documentation clutter.


Gene filtering

Hello,
I provided a pre-processed expression matrix for scdiff2 (and used -f norm ) with only the highly variable genes present, but based on the figures produced in the pre-run, it looks like more genes were filtered out. Does the pre-run filter out more genes, and is it possible to prevent or limit this?
Thanks,
Paraish

@paraish
Yes,  filtering highly_variable_genes is essential in prerun even the input matrix is normalized (-f norm) as it somehow de-noises the data.
We filter genes primarily based on the dispersion, We provide a parameter --min_disp to customize the cutoff, by default the minimum dispersion is 0.15, but you can set it as zero or even negative values if you want to keep more genes.  There is a plot "filter_genes_dispersion.pdf" inside the prerun results folder, you can use this plot to choose your dispersion cutoff (use the normalized plot on the left).
If you don't want the filtering at all, sure,  I provided another option --skipGeneFilter , you can use it to skip the filtering and keep all the genes.  use prerun -h for details.
Please update your program.

Thanks a lot - I'm still running some tests and will keep you posted.


#[serde(fallback = "name")] similar to #[serde(alias = "name")]

I'm in a bit of a conundrum. I have a pretty nasty set of data coming from an outside source that includes fields that should inherit other field's values if they're not present in the original data. I have quite a few fields like this.
Something like #[serde(fallback = "name")] would make life so much easier in this instance.
#[derive(Debug, Deserialize)]
#[serde(default, rename_all = "kebab-case")]
pub struct MyObnoxiousDataset {
    pub some_value: String,

    // Purposefully redundant, to illustrate multiple fallbacks and name vs Rust name, like #[serde(alias)]
    #[serde(fallback = "some-value", fallback = "some_value")]
    pub some_other_value: String,
}

impl Default for MyObnoxiousDataset {
    fn default() -> Self {
        Self {
            some_value: "some_value_default".to_string(),
            some_other_value: "doesn't matter because it'll just use some_value instead".to_string(),
        }
    }
}

let data: MyObnoxiousDataset = serde_json::from_str(r#"
    {
        "some-value": "foobar"
    }
"#);
println!("{}", data.some_other_value);
// Output: "foobar"

let data: MyObnoxiousDataset = serde_json::from_str(r#"
    {
        "some-value": "foobar",
        "some-other-value": "bazquux",
    }
"#);
println!("{}", data.some_other_value);
// Output: "bazquux"

Effectively, this would be similar to #[serde(alias = "name")] (similarly allowing for multiple fallbacks, preferably), but rather than completely remapping the field, it would allow the field to be deserialized into multiple fields as needed (including its original Rust name field).

I just found myself in the same need (and implementing Deserialize by hand). I'd prefer if the fallback name was not an explicit field of the struct though, but rather something handled internally by serde.
Other alternative I thought would work but doesn't is just being able to provide the same alias to different fields. I don't know if it's better to have a separate mechanism, maybe it helps give priority to aliases over fallbacks.

I just found myself in the same need (and implementing Deserialize by hand). I'd prefer if the fallback name was not an explicit field of the struct though, but rather something handled internally by serde.

This may already be possible, depending on your use case, with Default:
#[derive(Debug, Deserialize)]
#[serde(default, rename_all = "kebab-case")]
pub struct MyObnoxiousDataset {
    pub some_value: String,
    pub some_other_value: String,
}

impl Default for MyObnoxiousDataset {
    fn default() -> Self {
        Self {
            some_value: "some field's value".to_owned(),
            some_other_value: "the other field's default value".to_owned(),
        }
    }
}

[!IMPORTANT]
It's important to note that a Default will be constructed every time Serde deserializes, regardless of whether or not a default value is used.

The struggle is that there is currently no real way to have the Default value for a field fallback to another field in the JSON structure that's being deserialized.

Other alternative I thought would work but doesn't is just being able to provide the same alias to different fields.

That was kind of the idea behind fallback. It might not be the best name, but effectively it would operate in the same way, while providing multiple fields to fallback on, without changing the functionality / backwards compatibility with alias.
Not to shoot my own proposal in the foot (I'd love for the implementation to take this into account), but: One of the scenarios that this may or may not be able to take into account (which may actually be the more common use-case) is falling back to the value of an ancestor's or child's field. That probably won't be possible, even for this.
One workaround for all of this would be to have an interface for retrieving the values you need from your series of structs. That adds a bit of redundancy but leaves the deserialization to serde.
That might look like (mind the pseudo code):
struct cfg {
    pub sometree: OtherStruct,
    pub tree_two: SecondStruct,
}

struct OtherStruct {
    pub my_field: Option<String>,
}

struct SecondStruct {
   pub fallback_field: String,
}

pub fn get_my_field(): String {
    match CONFIG.sometree.my_field {
        Some(v) => v,
        None => CONFIG.tree_two.fallback_field,
    }
}

Again... kind of defeats half of the purpose of serde. But at least the half that you don't have to write is the deserialization, and instead the interface for your structs.

[!WARNING]
But, this may introduce performance issues, because you're moving this sort of logic from when the deserialization occurs to every time the field is read. That being said, if you're only reading that particular field once, or infrequently, then that may not matter.

@jmi2k

I just found myself in the same need (and implementing Deserialize by hand). I'd prefer if the fallback name was not an explicit field of the struct though, but rather something handled internally by serde.

This may already be possible, depending on your use case, with Default:
#[derive(Debug, Deserialize)]
#[serde(default, rename_all = "kebab-case")]
pub struct MyObnoxiousDataset {
    pub some_value: String,
    pub some_other_value: String,
}

impl Default for MyObnoxiousDataset {
    fn default() -> Self {
        Self {
            some_value: "some field's value".to_owned(),
            some_other_value: "the other field's default value".to_owned(),
        }
    }
}

[!IMPORTANT]
It's important to note that a Default will be constructed every time Serde deserializes, regardless of whether or not a default value is used.

The struggle is that there is currently no real way to have the Default value for a field fallback to another field in the JSON structure that's being deserialized.

Other alternative I thought would work but doesn't is just being able to provide the same alias to different fields.

That was kind of the idea behind fallback. It might not be the best name, but effectively it would operate in the same way, while providing multiple fields to fallback on, without changing the functionality / backwards compatibility with alias.
Not to shoot my own proposal in the foot (I'd love for the implementation to take this into account), but: One of the scenarios that this may or may not be able to take into account (which may actually be the more common use-case) is falling back to the value of an ancestor's or child's field. That probably won't be possible, even for this.
One workaround for all of this would be to have an interface for retrieving the values you need from your series of structs. That adds a bit of redundancy but leaves the deserialization to serde.
That might look like (mind the pseudo code):
struct cfg {
    pub sometree: OtherStruct,
    pub tree_two: SecondStruct,
}

struct OtherStruct {
    pub my_field: Option<String>,
}

struct SecondStruct {
   pub fallback_field: String,
}

pub fn get_my_field(): String {
    match CONFIG.sometree.my_field {
        Some(v) => v,
        None => CONFIG.tree_two.fallback_field,
    }
}

Again... kind of defeats half of the purpose of serde. But at least the half that you don't have to write is the deserialization, and instead the interface for your structs.

[!WARNING]
But, this may introduce performance issues, because you're moving this sort of logic from when the deserialization occurs to every time the field is read. That being said, if you're only reading that particular field once, or infrequently, then that may not matter.

Not to shoot my own proposal in the foot (I'd love for the implementation to take this into account), but: One of the scenarios that this may or may not be able to take into account (which may actually be the more common use-case) is falling back to the value of an ancestor's or child's field. That probably won't be possible, even for this.

This would actually be nice, but I think it doesn't necessarily undermines the usefulness of having a fallback field.

One workaround for all of this would be to have an interface for retrieving the values you need from your series of structs. That adds a bit of redundancy but leaves the deserialization to serde.

This is my current approach for other structs: A RawT for deserializing, which I refine into a T. Still, it would be nice to have a better alternative for this case in particular.


Removing underscore prefixes from private fields

Yested now follows a convention where private fields contains an underscore prefix.
I think it is highly discouraged. Thanks to the modern IDEs (especially IntelliJ), private fields are syntax highlighted differently from parameters, so there is no need to distinguish them with "_".
Moreover, the Kotlin documentation explicitly prohibit this:
never create a private property and call it _foo

http://kotlinlang.org/docs/reference/coding-conventions.html

Hi,
Thanks, didn't know that. Will fix it.
Jan
On 12. 1. 2015, at 8:54, "Balázs Bódi"<EMAIL_ADDRESS>wrote:
Yested now follows a convention where private fields contains an underscore
prefix.
I think it is highly discouraged. Thanks to the modern IDEs (especially
IntelliJ), private fields are syntax highlighted differently from
parameters, so there is no need to distinguish them with "_".
Moreover, the Kotlin documentation explicitly prohibit this:
never create a private property and call it _foo
http://kotlinlang.org/docs/reference/coding-conventions.html
—
Reply to this email directly or view it on GitHub
https://github.com/jean79/yested/issues/21.


Testing

This PR solves #8 with 11 tests currently.
It also adds a pre test configuration file src/setupTests.js due to an issue discussed in  - https://github.com/inrupt/solid-client-authn-js/issues/1676

Did you also have a look at https://github.com/SolidLabResearch/solid-web-app-template/ for the testing?

Did you also have a look at https://github.com/SolidLabResearch/solid-web-app-template/ for the testing?

I will do this now.

I have added a test for login authentication.
I will also add tests for error handling once #24  is merged

Starting the app gives the following warning:

I have made a separate issue for this as it is not related to this PR

I have made a separate issue for this one also


:connection_timeout random burst

Hello, i'm getting a rather odd error spam out of the blue here. About 5000 error events popped up in sentry after running an update, which resolved with a reboot:
11:30:31.085 [error] Task #PID<0.2538.0> started from #PID<0.2530.0> terminating
** (stop) exited in: GenServer.call(#PID<0.2525.0>, {:run_protocol, %ChromicPDF.Protocol{result_fun: nil, state: %{ignore_certificate_errors: false, offline: true, session_pool: [size: 10, timeout: 10000]}, steps: [call: &ChromicPDF.SpawnSession.create_browser_context/2, await: &ChromicPDF.SpawnSession.browser_context_created/2, call: &ChromicPDF.SpawnSession.create_target/2, await: &ChromicPDF.SpawnSession.target_created/2, call: &ChromicPDF.SpawnSession.attach/2, await: &ChromicPDF.SpawnSession.attached/2, call: &ChromicPDF.SpawnSession.set_user_agent/2, call: &ChromicPDF.SpawnSession.offline_mode/2, call: &ChromicPDF.SpawnSession.enable_page/2, await: &ChromicPDF.SpawnSession.page_enabled/2, call: &ChromicPDF.SpawnSession.blank/2, await: &ChromicPDF.SpawnSession.blanked/2, await: &ChromicPDF.SpawnSession.fsl_after_blank/2, output: &ChromicPDF.SpawnSession.output/1]}}, 5000)
    ** (EXIT) :connection_terminated
    (elixir 1.11.4) lib/gen_server.ex:1027: GenServer.call/3
    (chromic_pdf 0.7.2) lib/chromic_pdf/pdf/browser/channel.ex:21: ChromicPDF.Browser.Channel.run_protocol/3
    (chromic_pdf 0.7.2) lib/chromic_pdf/pdf/browser/session_pool.ex:110: ChromicPDF.Browser.SessionPool.do_init_worker/2
    (elixir 1.11.4) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir 1.11.4) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib 3.10) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Function: #Function<0.13638364/0 in ChromicPDF.Browser.SessionPool.init_worker/1>
    Args: []

11:30:31.209 [error] GenServer #PID<0.2550.0> terminating
** (stop) :connection_terminated
Last message: {:EXIT, #Port<0.63>, :normal}

It's weird because chromium wasn't updated D:
Perhaps the reconnect frequency could be toned down? I got 100s of these errors every second.

Hey @dabaer
the Supervisor docs say it has a max_restarts option (see here) which presumably could at least limit the number of restarts. But it doesn't seem to have a way to configure a "cooldown" period or so :/
Unfortunately I don't have much time currently to work on this. Is this a problem that occurs frequently for you or was it just once? Can you pinpoint why the Chromium process died? If this happens a lot, you could try the discard_stderr: false option to see Chrome's output.
Best,
malte

Hmm i'll have to keep an eye on it, and set that restart limit. I don't see any indication as to why, all I walked into was my Sentry instance screaming in agony that time and it hasn't ocurred since a reboot :(

it hasn't ocurred since a reboot :(

Well that's a good thing, right? 😄

@dabaer closing this for now, feel free to re-open if it happens again


node-gyp failed to rebuild

Pre-flight checklist

[X] I have read the contribution documentation for this project.
[X] I agree to follow the code of conduct that this project uses.
[X] I have searched the issue tracker for a bug that matches the one I want to file, without success.

Electron Forge version
7.2.0
Electron version
28.1.4
Operating system
13.0.1 (22A400)  M1
Last known working Electron Forge version
No response
Expected behavior
make  electron
Actual behavior

electron-forge make

✔ Checking your system
ber named 'AccessorSignature' in namespace 'v8'
typedef v8::Localv8::AccessorSignature Sig;
In file included from ../fsevents.cc:6:
../../../../nan/nan.h:2542:8: error: no matching member function for call to 'SetAccessor'
tpl->SetAccessor(
~~~~~^~~~~~~~~~~
/Users/tangsong/.electron-gyp/20.3.12/include/node/v8-template.h:814:8: note: candidate function not viable: no known conversion from 'imp::Sig' (aka 'int') to 'v8::SideEffectType' f…
void SetAccessor(
^
/Users/tangsong/.electron-gyp/20.3.12/include/node/v8-template.h:807:8: note: candidate function not viable: no known conversion from 'imp::NativeGetter' (aka 'void (*)(v8::Local<v8:…
void SetAccessor(
^
../fsevents.cc:14:10: fatal error: 'src/storage.cc' file not found
#include "src/storage.cc"
^~~~~~~~~~~~~~~~
3 errors generated.
make: *** [Release/obj.target/fse/fsevents.o] Error 1
Error: `make` failed with exit code: 2
at ChildProcess.onExit (/Users/tangsong/Documents/cqq/sop/electron_pension/node_modules/node-gyp/lib/build.js:203:23)
at ChildProcess.emit (node:events:513:28)
at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)

An unhandled rejection has occurred inside Forge:
Error: node-gyp failed to rebuild '/private/var/folders/6c/44_3vj9n74sdtr04s4nndkr40000gp/T/electron-packager/tmp-kxHxeG/Electron.app/Contents/Resources/app/node_modules/watchpack-chokidar2/node_modules/fsevents'
at ChildProcess.<anonymous> (/Users/tangsong/Documents/cqq/sop/electron_pension/node_modules/@electron/rebuild/lib/module-type/node-gyp/node-gyp.js:118:24)
  at ChildProcess.emit (node:events:513:28)

### Steps to reproduce

make a    electron project
"@electron-forge/cli": "^7.2.0",
  "@electron-forge/maker-deb": "^7.2.0",
  "@electron-forge/maker-rpm": "^7.2.0",
  "@electron-forge/maker-squirrel": "^7.2.0",
  "@electron-forge/maker-zip": "^7.2.0",
  "cross-env": "^7.0.3",
  "electron": "^28.1.4",
and use  electron-forge  to  make it

### Additional information

_No response_

Can you provide a minimal reproduction?


Getting started?

I don't think this is a "getting started" guide. It requires some understanding of mongodb, or at least databases in general. I made it past lesson 3, but I don't understand why and what is going on.
You might consider revising the description.

I'm not sure where it is noted that this is a getting started guide. Perhaps you could point me to it? Thanks!

Closing due to inactivity. Please feel free to reopen if needed


sample code should be using .net core

a .net core web app may have been nice to see instead of the outdated sample framework

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 906907ec-cbe2-59b1-a84e-ce1b1343756f
Version Independent ID: f498962c-f738-ccd8-a619-de854fac37da
Content: Tutorial: Enable authentication in a web application - Azure AD B2C
Content Source: articles/active-directory-b2c/tutorial-web-app-dotnet.md
Service: active-directory
Sub-service: b2c
GitHub Login: @msmimart
Microsoft Alias: mimart

@msmimart Thank you for your feedback . We will pass this feedback to the team.

@274188A  Thanks for your feedback, we will discuss this with the content team, meanwhile you can have a look at this URL which talks about a sample .Net Core app :
https://github.com/Azure-Samples/active-directory-aspnetcore-webapp-openidconnect-v2/tree/master/4-WebApp-your-API/4-2-B2C

@274188A  We will be closing this now, feel free to tag me in your comment if you have any questions

Also, this isn't even a tutorial. This is a link to sample code.


Check if a path of a certificate was specified

A status of a certificate was not determined correctly. Before
this commit, the property method has_certificate() used to
return True even if the certificate was not declared. The
exception TypeError (stat: path should be string, bytes,
os.PathLike or integer, not NoneType) was raised while the
fuzzer was trying to check if a path to the certificate
is a valid file.
https://github.com/SAP/odfuzz/blob/184660b47b2e17838b39922cc17e40aa2304a985/odfuzz/fuzzer.py#L1170-L1172

Codecov Report

Merging #19 into master will decrease coverage by 0.02%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master      #19      +/-   ##
==========================================
- Coverage   31.58%   31.56%   -0.03%
==========================================
  Files          15       15
  Lines        3134     3136       +2
==========================================
  Hits          990      990
- Misses       2144     2146       +2

Impacted Files
Coverage Δ

odfuzz/config.py
55.1% <0%> (-2.35%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 184660b...8a935f1. Read the comment docs.

Funny that there is a fix for something we found on Friday and had problems with (used on HTTP service on SCP, which should not need certificate) - and PR is already here when I just wanted to create proper issue :)


42seoul

for teacher

@taeng42 Thank you for your patience while waiting for a review and processing your request. Pull request merged.
Please start requesting the licenses in about an hour, to let the changes to propagate through our system.


Use multi selection in recyclerview in fragment

Not an issue but i really need help, i managed to use flexible adapter in my project but following instructions on how to make the item selectable didnt work for me, tried the selection modes wiki and the ActionModeHelper Wiki and i'm really lost. Hope you can understand, spanish is my native language so it will be some mistakes in my grammar
First of all my "item" its like this
public class TrazaFlex extends AbstractFlexibleItem<TrazaFlex.ViewHolderTrazaFlex> {
   public TrazaFlex(){

    }

...
...
   @Override
    public void bindViewHolder(FlexibleAdapter<IFlexible> adapter, ViewHolderTrazaFlex holder, int position, List<Object> payloads) {
        holder.mTitle.setText(nombreTraza);
        // Title appears disabled if item is disabled
        holder.mTitle.setEnabled(isEnabled());

        Context context = holder.itemView.getContext();
        Drawable drawable = DrawableUtils.getSelectableBackgroundCompat(
                Color.WHITE, Color.parseColor("#dddddd"), // Same color of divider
                DrawableUtils.getColorControlHighlight(context));
        DrawableUtils.setBackgroundCompat(holder.itemView, drawable);
    }

   public class ViewHolderTrazaFlex extends FlexibleViewHolder {
        public TextView mTitle;

        public ViewHolderTrazaFlex(View view, FlexibleAdapter adapter) {
            super(view, adapter);
            mTitle = (TextView) view.findViewById(R.id.id_nombre_traza);
        }
    }

Tried  to follow the ActionModeHelperWiki and the SelectionModeWiki. And my adapter its like this:

public class TrazaFragment extends Fragment implements ActionMode.Callback, FlexibleAdapter.OnItemClickListener, FlexibleAdapter.OnItemLongClickListener {

    ArrayList<Traza> listaTrazas;
    RecyclerView recyclerTrazas;
    View vista;

    private FlexibleAdapter mAdapter;
    private ActionMode mActionMode;

    @Nullable
    @Override
    public View onCreateView(@NonNull LayoutInflater inflater, @Nullable ViewGroup container, @Nullable Bundle savedInstanceState) {
        getActivity().setTitle("Trazas");
        vista = inflater.inflate(R.layout.fragment_traza, container, false);
        listaTrazas = new ArrayList<>();
        recyclerTrazas = (RecyclerView) vista.findViewById(R.id.recyclerTrazas);
        recyclerTrazas.setLayoutManager(new LinearLayoutManager(getContext()));

        GetTrazas mAttempTraza = new GetTrazas();
        mAttempTraza.execute((Void) null);

        return vista;
    }

    @Override
    public boolean onCreateActionMode(ActionMode mode, Menu menu) {
        mode.getMenuInflater().inflate(R.menu.selected_menu, menu);
        mAdapter.setMode(Mode.MULTI);
        return true;
    }

    @Override
    public boolean onPrepareActionMode(ActionMode mode, Menu menu) {
        return false;
    }

    @Override
    public boolean onActionItemClicked(ActionMode mode, MenuItem item) {
        return false;
    }

    @Override
    public void onDestroyActionMode(ActionMode mode) {
        mAdapter.setMode(Mode.IDLE);
        mActionMode = null;
    }

    @Override
    public boolean onItemClick(View view, int position) {
        if (mActionMode != null && position != RecyclerView.NO_POSITION) {
            // Mark the position selected
            toggleSelection(position);
            Toast.makeText(getContext(),"OnItemLongClick ", Toast.LENGTH_LONG).show();
            return true;
        } else {
            Toast.makeText(getContext(),"OnItemLongClick ", Toast.LENGTH_LONG).show();
            // Handle the item click listener
            // We don't need to activate anything
            return false;
        }
    }

    @Override
    public void onItemLongClick(int position) {
        if (mActionMode == null) {
            getActivity().startActionMode(this);
        }
        Toast.makeText(getContext(),"OnItemLongClick ", Toast.LENGTH_LONG).show();
        toggleSelection(position);
    }

    /**
     * Toggle the selection state of an item.
     * If the item was the last one in the selection and is unselected, the ActionMode
     * is stopped.
     */
    private void toggleSelection(int position) {
        // Mark the position selected
        mAdapter.toggleSelection(position);

        int count = mAdapter.getSelectedItemCount();

        if (count == 0) {
            mActionMode.finish();
        } else {
            setContextTitle(count);
        }
    }
    private void setContextTitle(int count) {
        mActionMode.setTitle(String.valueOf(count) + " " + (count == 1 ?
                "Action Selected One" :
                "Action Selected Many"+count));
    }

    public class GetTrazas extends AsyncTask<Void, Void, Boolean> {
      //connects to service and get json
            return null;
        }
        @Override
        protected void onPostExecute(final Boolean success) {

            recyclerTrazas = (RecyclerView) vista.findViewById(R.id.recyclerTrazas);
////            recyclerTrazas.setLayoutManager(new LinearLayoutManager(getContext()));
            List<IFlexible> listaTrazasFlex = getDatabaseList();
            FlexibleAdapter<IFlexible> adapterFlexTraza = new FlexibleAdapter<>(listaTrazasFlex);
            recyclerTrazas.setAdapter(adapterFlexTraza);

        }
        public List<IFlexible> getDatabaseList() {
            //connects to json and returns a list

                }
            }
            catch (JSONException e) {

            }
            return list;
        }
    }

}

Already saw the example app that you provide, because you did exactly the thing i need to do, but it confuses me. Hope you can help me

@nexusblake, and what is not working?
It looks similar, as you said, of what I did. It should be fine (I just used the support ActionMode).

Documentation is available in Wiki pages: either you implement the ActionMode from scratch (similar to what you were doing) either you use ActionModeHelper. Wiki has both methods.


Add temporary refresh of build page when build is finished

This polls the api and refresh when the build gets a result.
Depends on #149

The flake8 warning is alredy fixed in #149

This is a temporary solution until the react frontend is ready.


WIP: Improve flexibility of message bus

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thank you! This looks good. Can you also make it do the same for deserializing messages?

@ejsmith I'm on a way to this. Thank you!

@ejsmith I think that we should use IBasicProperties to store messageType instead of MessageBusData but it's breaking change

I would generally agree with that change. Not sure if it’s worth a breaking change though. Maybe we start by storing the type in both spots?

On deserialization we would need to check in both spots for it.

I'm going to get this merged shortly and I'm going to go ahead and get the message type pulled out of the message body into a property as well.

@ejsmith I'm would be happy to that feature. I'm sorry that I don't find time to do it myself


$patch remove keyword from required

I saw in the tests how to add an attribute to the required keyword  { "op": "add", "path": "/required/-", "value": "q" }
but I want to ask how can i remove a keyword from being required I tried
{ "op": "remove", "path": "/required/-q"} but it did not work i seem to get it wrong so if anyone can help I would be grateful.
Thank you

Value-based array operations are not supported yet is JSON patch. See this issue: https://github.com/json-patch/json-patch2/issues/18
You need to remove values in an array based on index.
An example target JSON document:
   { "foo": [ "bar", "qux", "baz" ] }

A JSON Patch document:
   [
     { "op": "remove", "path": "/foo/1" }
   ]

The resulting JSON document:
   { "foo": [ "bar", "baz" ] }


Draft implementation of algebraic enum without content attribute

Draft implementation of support for Algebraic enums without specifying the content attribute (see https://github.com/1Password/typeshare/issues/141).
Only Unit variants and anonymous structs variants are supported (tuple variants explicitly require the content attribute), and I built support for Kotlin, Scala, Swift and TypeScript output.
This was done by adding a new RustEnum type (AlgebraicFlattened, the name is not great, I welcome any other suggestion) without the content_key.

This is great! Any plans to get this in?
Additionally, would this support serde rename?
#[typeshare]
#[serde(tag = "type")]
pub enum SomeEnum {
    A,
    B { field1: String },
    #[serde(rename = "Q")]
    C { field1: u32, field2: f32 },
    D { field3: Option<bool> },
}

Should generate a type like
export type SomeEnum =
	| { type: "A" }
	| {
            type: "B",
	    field1: string;
          }
	| {
            type: "Q", // < has been renamed
	    field1: number;
	    field2: number;
          }
	| {
            type: "D",
	    field3?: boolean;
          };

Hi this is very cool! Any progress on this? I could also try to help out :)

No sign of interest from the maintainers in 6 months; Tired of seeing this in my open PRs. Closing.


Controller logs should include timestamps

At the moment controller logs don't include timestamps which makes them not very helpful. This is not an issue if docker is configured to log to journald, but if it is configured with the json-file driver, then log entries fetched by docker logs or kubectl logs don't include any timestamps.
Thus controller logs (builder's logs also have this problem) should include timestamp, just like the router or registry do.
I'm using Controller v2.11.0.

If you're fetching the logs you can use the --timestamps option flag like kubectl -n deis logs --timestamps <deis-controller-pod>. Kubectl hides timestamps by default for whatever reason.

@bacongobbler That works. docker logs still doesn't show timestamps, but it doesn't matter with the kubectl --timestamps. Thank you.


No documented way to import MarkdownEngine

I was looking for a reference to the instance of markdown-it after all plugins are configured so I can use it to render markdown in my printing extension instead of duplicating the rendering pipeline.
Rummaging through the codebase I found MarkdownEngine and its render method, and this is exactly what I need but there is no mention of this in the API page and doesn't seem to be any reliable way to reference it.
If this actually is supported and I just failed to RTFM then feel free to be ironic at me in my Stack Overflow question on this topic.
I think the place for it is as a property vscode.env.markdownEngine alongside the clipboard object.

MarkdownEngine is part of the markdown extension's implementation. It is not public api and there is no way for you to consume it.
We could add a command that takes a markdown string and returns html. Is that what you need?

a command that takes a markdown string and returns html

Yes, that would be very suitable.
To clarify, the use-case: I wrote an extension that adds printing. One of the features is that you can print rendered markdown. I was loading my own instance of markdown-it but it would be enormously preferable to have VS Code perform the rendering so that any markdown extensions in force for the preview are also in force for printing.
There will be some time before this makes it to release copies of VS Code, so in the interim I've figured out a nasty little hack that gets me a reference to the internal instance of markdown-it. I look forward to retiring it.

Command is now called markdown.api.render and we require that you either pass a string or a vscode document (we won't automatically use the active editor)


Pennsylvania County-Level Official Scraper

Good afternoon!  I was hoping to have someone take a look at my code for the ported Pennsylvania scraper and offer feedback based on requirements, coding practices, or domain-specific knowledge.
At this point, the scraper includes all data from the cmdc-tools scraper, with the notable exception of ventilator data – the respective variables do not appear to have been added to meta.covid_variables yet.
I was specifically hoping that someone could take a look at my mappings of argcis columns to meta.covid_variables.  As this is my first scraper, I wanted to make sure that the mappings I've made are congruent with the definitions provided on the Pennsylvania dashboard info page.

Awesome, thanks @christopherturner .
I will look at this more closely tomorrow. At first glance I notice that the code is very clean with great style. We love that!

Additionally, the following is the reference that I used to determine the measurements and units that my code reflects.  On the Pennsylvania Data website, it's stated that:

Total cases, negatives, and recovered represent unique individuals. Total PCR Tests can include numerous tests for the same individual.
Total case counts include confirmed and probable cases.
Negative case data only includes negative PCR tests. Negative case data does not include negative antibody tests.
Individuals who have recovered is determined using a calculation, similar to what is being done by several other states. If a case has not been reported as a death, and it is more than 30 days past the date of their first positive test (or onset of symptoms) then an individual is considered recovered.

This is looking good, thanks @christopherturner
I need to see why the tests are failing. I think, but don't know for sure yet, that the issue is your fork of this repo doesn't have permissions to push the docker container for postgres to this repo's set of docker containers :thinking:
I'll take a look though

Hey @christopherturner  I took another look and found a few things:

The "county" column from the data source includes an entry for Pennsylvania, the whole state. I filter that out
I needed to make a change to the DatasetBase class to compare county names and other strings using all lowercase letters -- there were some errors with captialization. Totally not in scope for your PR, but it fixed a test so I wanted to point out why I did it in case you look at my changes and wonder what I did
I commented out the probable cases variable. That isn't a very precise variable and we found that it wasn't very useful so we stopped recording it.

There is one more change we need to figure out...
For the hospital scraper the county column also includes things like hospital regions. We need to come up with a solution for how to remove those and only keep data for counties. @cc7768 any ideas?

Note, the hospital regions include
Out[19]:
array(['East Central HCC', 'HCC of Southwest PA', 'Keystone HCC',
       'North Central HCC', 'Northeast', 'Northcentral', 'Northeast HCC',
       'Northern Tier HCC', 'Northwest', 'Southcentral', 'Southeast HCC',
       'Southeast', 'Southwest'], dtype=object)

On the Florida scraper, I noticed that some rows were dropped based on a query like this: .query("location not in (12998, 12999)").  My most basic approach would be excluding those hospital regions, based on the above keys, using a query like this.  Looking at the ArcGIS table, I don't see any values where we could expect counties to be categorically different from hospital regions (it seems most everything is non-null in both cases).

@cc7768 reminded me today that  we actually have room in the schema for collecting data by hospital region.
I have a change in the works that will make it easier to add new locations like this, fix the test runner issues here, and make it easier to test locally. Should be in tomorrow. Stay tuned and I'll work with you to add the hospital region data!

Hey @christopherturner the updates to CI and database interactions are done now.
If we can merge in the main branch into this one we should be able to pick those changes up. Are you comfortable doing that merge here?

Sounds great, @sglyon!  And definitely – I can pull the upstream and merge it into my branch/PR!

@christopherturner thanks for doing the merge.
I pushed one commit that added the Penn scrapers to the __init__.py  files and now we have a failing test because we haven't taught the database about the hospital regions.
@cc7768 do you think it is worthwhile to add these hospital regions to the database and collect data for them? Or should we follow @christopherturner suggestion to filter them out like we do non-counties in FL?

Let's go ahead and filter them out for now and work to get this merged and live.
We can re-visit this (and other states with hospital regions) in the future

Thanks for all of the feedback @sglyon and @cc7768!  In my latest commit, I've added the array of non-county regions and filtered out any rows that contain those values in the location_name column.
I've also moved the filter for location_name != "Pennsylvania" into that logic, rather than filtering it out in the return statement of PennsylvaniaHospitals.pre_normalize().

Great work @christopherturner !


Add extra slot to data table - allow level-left to be overwritten

Allow this number - http://d.pr/i/Msi7MB to be replaced with action buttons.
Here is how the table will look with action buttons - http://d.pr/i/lcF6uo
Please ping me if I have to do some extra changes here.

@rafaelpimpa - Please review this when you have time.
Ping @jtommy

@tgosp Thanks. We're updating docs as you can see. We'll review as soon as possible.

Thank you so much @jtommy
I'm missing few more things around the data component.
Expect more requests soon.

Hi @tgosp, thanks for contributing!
We gonna make it a lot simpler, just a simple named slot so the dev can add anything he/she wants, that's why this will be closed, thanks for the original idea!


fix client eslint config - missing plugins - Beginners Only Please!

Here is a hint to help you fix your first bug on an open source project. Look in the client directory at the eslintrc.json file. Compare this with the devDependencies in the client>package.json and add the eslint plugins to the to the eslintrc.json file.
Good luck!

I would like to take this up, it would help me initiate the process of figuring out configuration. But I am still struggling with the setup. I followed the instructions from Readme in client directory and getting errors. Need help to find what I am missing.

@khusbuchandra what issues are you having? With git triangle workflow setup? If that is all good, the next step is to run npm install at:

the root of the project folder - for server node modules
cd into client and then run npm install - for preact node modules

After that you will need to setup a secrets.json file in config folder. Copy secrets-example.json. You can use the mongodb setup as is for now but will eventually need to get a google api key and update that section accordingly. Hope that helps. Feel free to ask specific questions.

@khusbuchandra you can also hit me up on slack and I can walk through things with you to get you started.

Thanks @jkwening  & @motosharpley , After few trial and errors, I figured it out. Will start contributing now.


Life Cycle Methods Vs useEffect Hook in React

Proposed title of article
Life Cycle methods Vs useEffect Hook in React
Introduction paragraph (2-3 paragraphs):
Having to constantly switch between functions, classes, higher-order components, and render props in a react project could be so tricky to wrap your head around. The react core team came up with the hooks which make us write reusable logic seamlessly. In this article we will explore how to replace the life cycle methods with the useEffect hook.
Article Structure

Component Lifecycle methods
useEffect Hook
useEffect Vs componentDidMount
useEffect Vs componentDidUpdate
Conclusion

Key takeaways:

What are the 3-5 most important things the reader should understand or be able to do after reading this article? Use this area to get your ideas down on the bulk of your article or tutorial.

The reader at the end of the article will be able to:

Understand how to use the useEffect hook effectively
Learn how the useEffect hook replaces componentDidMount and componentDidUpdate
Learn from code examples how to migrate a class-based component lifecyle to functional component useEffect
Get tips on how to use dependencies when using the useEffect especially when calling API endpoints

References:

Please list links to any published content/research that you intend to use to support/guide this article.

React Documentation

Hi, @Umoren. Thanks for submitting your topic suggestion. Can you please fill out the remaining sections of the topic suggestion form I've added back in for you?

Hi @louisefindlay23. I've updated the topic suggestion form.

Oh didn't see this.
It covers most of my outline except the dependencies part.
On Thu, Sep 2, 2021, 12:21 PM Louise @.***> wrote:

Thanks, @Umoren https://github.com/Umoren. Can you tell me how your
article will differ from existing articles on the same topic such as this
LogRocket
https://blog.logrocket.com/lifecycle-methods-with-the-useeffect-hook/
guide?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/section-engineering-education/engineering-education/issues/3529#issuecomment-911556632,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFMHS6LQU6A7JWEPGGIYPYLT75M2RANCNFSM5CY6HQEQ
.

@Umoren, we focus on original and unique content so I would suggest you expand your proposed article to cover additional adjacent topics or tell me how you will be write about a unique viewpoint or take on this. Alternatively, choose a different topic which has not been covered extensively elsewhere.

Thank you for the topic suggestion @Umoren , however this is a bit outside of our preferred topic areas. We're more interested in original, practitioner-focused content that takes a deeper dive into programming-centric concepts. Please feel free to suggest an alternate topic to explore. cc @louisefindlay23


🐛 Bug Report: Split Monorepo projects as an individual backstage entity

📜 Description
I would like to split monorepos and treat each component of the monorepo as an individual Backstage entity and  added below mentioned catalog-info.yaml file at the root of the monorepo project
---
apiVersion: backstage.io/v1alpha1
kind: Location
metadata:
  name: "Dauntless Unified Experience - TEST"
spec:
  type: url
  targets:
    - ./apps/claims-finder-snapshot/backstage.yaml
    - ./apps/claims-viewer/backstage.yaml
    - ./apps/ui-kds-angular-sandbox/backstage.yaml

and below mentioned yml at the root of the individual project inside ./apps/ directory
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: claims-finder-snapshot
  title: Claims Finder Snapshot
  annotations:
    backstage.io/techdocs-ref: dir:.
spec:
  type: library
  lifecycle: experimental
  owner: unknown

However, I am not yet seeing the individual projects as separate entities under Catalog.
Can somebody please help me how can I split monorepo project as individual entities in Backstage?
👍 Expected behavior
I would like to split monorepos and treat each component of the monorepo as an individual Backstage entity. A split monorepo will have multiple associated catalog entries, one for each sub-component within the monorepo. It will contain many Backstage metadata files and many sets of TechDocs.
👎 Actual Behavior with Screenshots
I am not seeing the individual projects as separate entities under Catalog and not even monorepo as single entry under Catalog
👟 Reproduction steps
Please see description for the code
Steps:

Take any monorepo project.
Add below mentioned catalog-info.yaml file at the root of the project

---
apiVersion: backstage.io/v1alpha1
kind: Location
metadata:
 name: <name of the monorepo>
spec:
 type: url
 targets:
   - <list of inidividual yaml file for Component entities>

Add below mentioned catalog-info.yaml file at the root of the individual project

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: claims-finder-snapshot
  title: Claims Finder Snapshot
  annotations:
    backstage.io/techdocs-ref: dir:.
spec:
  type: library
  lifecycle: experimental
  owner: unknown

Start the server and go to /catalog.

📃 Provide the context for the Bug.
No response
🖥 Your Environment
cn330348@CNCL4M9DWFH99 devx-backstage % yarn backstage-cli info
OS:   Darwin 22.5.0 - darwin/arm64
node: v16.20.0
yarn: 3.5.1
cli:  0.22.7 (installed)
backstage:  1.14.1

Dependencies:
  @backstage/app-defaults                            1.3.1
  @backstage/backend-app-api                         0.4.2, 0.4.3
  @backstage/backend-common                          0.18.4, 0.18.5
  @backstage/backend-dev-utils                       0.1.1
  @backstage/backend-plugin-api                      0.5.1, 0.5.2
  @backstage/backend-tasks                           0.5.1, 0.5.2
  @backstage/backend-test-utils                      0.1.37
  @backstage/catalog-client                          1.4.1
  @backstage/catalog-model                           1.3.0
  @backstage/cli-common                              0.1.12
  @backstage/cli-node                                0.1.0
  @backstage/cli                                     0.22.7
  @backstage/config-loader                           1.3.0
  @backstage/config                                  1.0.7
  @backstage/core-app-api                            1.8.0
  @backstage/core-components                         0.12.5, 0.13.1
  @backstage/core-plugin-api                         1.5.1
  @backstage/dev-utils                               1.0.15
  @backstage/errors                                  1.1.5
  @backstage/eslint-plugin                           0.1.3
  @backstage/integration-aws-node                    0.1.2, 0.1.3
  @backstage/integration-react                       1.1.13
  @backstage/integration                             1.4.5
  @backstage/plugin-api-docs                         0.9.3
  @backstage/plugin-app-backend                      0.3.45
  @backstage/plugin-auth-backend                     0.18.3
  @backstage/plugin-auth-node                        0.2.13, 0.2.14
  @backstage/plugin-catalog-backend-module-gitlab    0.0.0-use.local
  @backstage/plugin-catalog-backend-module-ldap      0.5.12
  @backstage/plugin-catalog-backend                  1.9.1
  @backstage/plugin-catalog-common                   1.0.13
  @backstage/plugin-catalog-graph                    0.2.30
  @backstage/plugin-catalog-import                   0.9.8
  @backstage/plugin-catalog-node                     1.3.6
  @backstage/plugin-catalog-react                    1.6.0
  @backstage/plugin-catalog                          1.11.0, 1.11.1
  @backstage/plugin-cicd-statistics-module-gitlab    0.0.0-use.local
  @backstage/plugin-cicd-statistics                  0.1.20
  @backstage/plugin-github-actions                   0.5.18
  @backstage/plugin-kubernetes-backend               0.11.0
  @backstage/plugin-kubernetes-common                0.6.3
  @backstage/plugin-kubernetes                       0.9.1
  @backstage/plugin-org                              0.6.8
  @backstage/plugin-permission-common                0.7.5
  @backstage/plugin-permission-node                  0.7.7, 0.7.8
  @backstage/plugin-permission-react                 0.4.12
  @backstage/plugin-proxy-backend                    0.2.39
  @backstage/plugin-scaffolder-backend-module-gitlab 0.0.0-use.local
  @backstage/plugin-scaffolder-backend               1.14.0
  @backstage/plugin-scaffolder-common                1.3.0
  @backstage/plugin-scaffolder-node                  0.1.3
  @backstage/plugin-scaffolder-react                 1.4.0
  @backstage/plugin-scaffolder                       1.13.1
  @backstage/plugin-search-backend-module-catalog    0.1.1
  @backstage/plugin-search-backend-module-pg         0.5.6
  @backstage/plugin-search-backend-module-techdocs   0.1.1
  @backstage/plugin-search-backend-node              1.2.1
  @backstage/plugin-search-backend                   1.3.1
  @backstage/plugin-search-common                    1.2.3
  @backstage/plugin-search-react                     1.6.0, 1.6.1
  @backstage/plugin-search                           1.3.0
  @backstage/plugin-tech-radar                       0.6.4
  @backstage/plugin-techdocs-addons-test-utils       1.0.14
  @backstage/plugin-techdocs-backend                 1.6.2
  @backstage/plugin-techdocs-module-addons-contrib   1.0.13
  @backstage/plugin-techdocs-node                    1.7.1
  @backstage/plugin-techdocs-react                   1.1.6
  @backstage/plugin-techdocs                         1.6.2, 1.6.3
  @backstage/plugin-user-settings                    0.7.3
  @backstage/release-manifests                       0.0.9
  @backstage/test-utils                              1.3.1
  @backstage/theme                                   0.2.19, 0.3.0
  @backstage/types                                   1.0.2
  @backstage/version-bridge                          1.0.4

👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
None

Is there any logs in the backend that can help us here? I have a feeling that it's probably because Location entities might not be allowed to be registered and you will need to enable that for that particular host. Should be some more information in the logs.

@poonamgp14
Check this config file
---
apiVersion: backstage.io/v1alpha1
kind: Location
metadata:
  name: "Dauntless Unified Experience - TEST"
spec:
  type: url
  targets:
    - ./apps/claims-finder-snapshot/backstage.yaml
    - ./apps/claims-viewer/backstage.yaml
    - ./apps/ui-kds-angular-sandbox/backstage.yaml

I think this is not a valid
Please use entity validator to check.
I get this error while doing the same
Error: "metadata.name" is not valid; expected a string that is sequences of [a-zA-Z0-9] separated by any of [-_.], at most 63 characters in total but found "Dauntless Unified Experience - TEST".

@benjdlambert Yes, I am getting below mentioned error that Location entity is not allowed
Entity location:default/dauntless_unified_experience_test at url:https://gitlab.centene.com/centene-developer-experience/dauntless-unified-experience-test/-/blob/develop/catalog-info.yaml, originated at url:https://gitlab.centene.com/centene-developer-experience/dauntless-unified-experience-test/-/blob/develop/catalog-info.yaml, is not of an allowed kind for that location type=plugin entity=location:default/generated-bc39469d4f96b1fbe35fbab1db7cc9d6786127bc location=url:https://gitlab.centene.com/centene-developer-experience/dauntless-unified-experience-test/-/blob/develop/catalog-info.yaml

How can I enable Location entity?

Adding Location to the app-config.yaml file worked and I am able to split up monorepo. Thanks for the suggestion

Nice glad it worked out! Going to close this as fixed!


Add ability to benchmark dynamic SUT

Now users should be able to run a benchmark for any registered ModelGauge SUT via the -s option. Also add some very basic defaulting for SUT static content so that reports will render.

Yeah, honestly, I think we could drop suts.toml entirely, in that the SutDescription has a name field that nobody is messing with. But let's see where things go.


On hover bubble works as hole to show underlying background

Hello I have tried and tried to work out if this is possible using particle.js
I want an image as a background and a white layer over this that when you hover over the particle layer the bubbles show image underneath a bit like this.
https://revolution.themepunch.com/particle-effect-for-wordpress/
any help would help me keep little hair I have left lol
Kind regards
Mike

Original project issue https://github.com/VincentGarreau/particles.js/issues/428 thanks to @Mickperks

The original project is abandoned, I'll have a look on it, it seems really cool

This feature will be released in the next version

Hello
That sounds excellent
any idea when this would be
Mike
On Wed, 11 Mar 2020 at 23:34, Matteo Bruni<EMAIL_ADDRESS>wrote:

Closed #117 https://github.com/matteobruni/tsparticles/issues/117 via
#122 https://github.com/matteobruni/tsparticles/pull/122.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/matteobruni/tsparticles/issues/117#event-3120906930,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOYSMLYQDBF3GHNE6ZEWG3DRHANXRANCNFSM4LFQJOHA
.

It's already live
You can checkout the codepen sample here: https://codepen.io/matteobruni/pen/yLNvqxG

wow excellent work
what did you have to do?
amazing and thank you your the man!
:)
KInd regards
Mike
On Thu, 12 Mar 2020 at 08:30, Matteo Bruni<EMAIL_ADDRESS>wrote:

It's already live
You can checkout the codepen sample here:
https://codepen.io/matteobruni/pen/yLNvqxG
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/matteobruni/tsparticles/issues/117#issuecomment-598065592,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOYSML6NN7QGKBID6EK76FTRHCMTBANCNFSM4LFQJOHA
.

It was not so hard, the HTML canvas already has this feature, I activate it when the config needs it.
Hope it works fine, if you need help I'm here

Hi
I have been using your particle.js json file with smartslider 3
which woks really well i have let them know you have updated this
be great if could get to work in there
kind regards
Mike
On Thu, 12 Mar 2020 at 08:47, Matteo Bruni<EMAIL_ADDRESS>wrote:

It was not so hard, the HTML canvas already has this feature, I activate
it when the config needs it.
Hope it works fine, if you need help I'm here
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/matteobruni/tsparticles/issues/117#issuecomment-598071803,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOYSML5UWVAT2SN5NKO3PR3RHCOSLANCNFSM4LFQJOHA
.


Feature/grid

CSS grid layout with borders to mark off sections for the features.

Notes

each grid is coloured for clarity right now