File Share Snapshot

@derekbekoe @tjprescott ping me when you're available. We need to review this together.

Can you get it to build correctly so I can review?

I ran on both profiles.

Rebase and waiting for the CI.


Why searching a VM by name is so difficult?

I have been using pyvmomi to automate VM operation for a while now. When search for the VM as the automation target, I couldn't help but think, finding the correct target VM by just providing VM names from vCenter via pyvmomi is made very difficult with a lot of restrictions.
FindByDNSName and FindByIP requires VMs powered on and have Tools running reporting back the info. Meaning they are not applicable for powered off VMs.
FindByUuid requires, well, the UUID of the VM, which no body with healthy mental state will remember VMs by UUIDs.  To find the UUID, you need to find the VM from vCenter, powerCLi, pyvmomi or login to the OS, get and save the UUID. Unless you always targets the same set of VM, this method seems very counter-productive.
FindByDatastorePath and FindByInventoryPath, they both require the user to know additional information about the VM which they don't really care on day to day operations. They are also counter-productive, since you need have another query to find out either the datastore or inventory path.
Then you left with FindChild to actually search by VM name, which only searches the immediate child object. providing Datacenter is not good enough, since host is considered the child, and VM in a DRS cluster could happen to be on any of the hosts.  Either you do a recursive search through the entities, or the less civilized way of get the entire content and use for loop. On a busy vCenter, with 5000 VMs, it take around 5 - 7 seconds to get back to find a single VM.  When you want the script to search multiple vCenters, then you are looking at a good 30 - 40 sec just to find the VM. Not to mention the actual work tot he VM is not even started.
Now, in comparison with the HTML5 web client on vCenter, when you search for a VM with name, you don't need to provide anything else but the name, and the result returns almost immediately.
Why pyvmomi could not offer the function of search for a VM, powered on or off, by just providing the VM name?

I agree, there isn't any good API to call to get a VM by name.
I think the fastest way is to fetch all the VM 'name' property under a datacenter (assuming you know the datacenter), and then iterating that.
I have a gist that fetches more than VM name (https://gist.github.com/prziborowski/ba3ebf610dd6cca3f4e7be5e2874499f), but you could remove that if you want. But if you want to find VMs that are powered on or off, then fetching runtime (or runtime.powerState) might be what you need.
My example lacks filtering on datacenter. That would be a problem if multiple VMs have the same name (which could still be a problem filtering on datacenter as the VMs could be in different VM folders).
pyvmomi can't do what the html5 web client does as the web client have a server process where VC is. So it can do things this API can't.


If Sanitastiion Enabled In HudiStreamer It is taking too much time

Tips before filing an issue
Describe the problem you faced
I have enabled the SANITIZE_SCHEMA_FIELD_NAMES hudiDeltaStreamer is stuck after reading CSV.
I think we can refactor the code it too better way.
Instead of using withColumnRenamed the transformation should be something like this
def transformSchemaBeginEndCharReplace(spark: SparkSession, final_stream: Dataset[Row], pii_masking_col: Seq[Any]): Dataset[Row] = {
val sql_select = new StringBuilder
val schema = final_stream.schema
for (i <- schema) {
if (i.dataType.isInstanceOf[StructType] || i.dataType.isInstanceOf[ArrayType]) {
sql_select.append(s"cast(to_json(${i.name}) as String)")
sql_select.append(" as ")
sql_select.append(avroSchemaNameConversionBeginEndCharReplace(i.name) + " , ")
  }
  else if (pii_masking_col.contains(i.name)) {
    sql_select.append(s"sha1(`${i.name}`)")
    sql_select.append(" as ")
    sql_select.append(avroSchemaNameConversionBeginEndCharReplace(i.name) + " , ")
  }
  else {
    sql_select.append(s"`${i.name}`")
    sql_select.append(" as ")
    sql_select.append(avroSchemaNameConversionBeginEndCharReplace(i.name) + " , ")

  }
}
val final_sql = sql_select.toString().stripSuffix(" , ").split(",")
final_stream.selectExpr(final_sql: _*)

}
def avroSchemaNameConversionBeginEndCharReplace(name: String) = {
val regexPattern = "(^[0-9])|(^[^a-zA-Z_])|(([^A-Za-z0-9_])$)|([^A-Za-z0-9_])".r
val outputString = regexPattern.replaceAllIn(name, m => {
if(m.group(1)!=null){
s"${m.group(1)}"
}
else if(m.group(2)!=null || m.group(3) != null ){
""
}
else {
""
}
})
outputString

}
We can set and adjust this work faster for my local transformation
To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.
Environment Description

Hudi version : 0.14.1

Spark version : 3.3

Hive version :

Hadoop version :

Storage (HDFS/S3/GCS..) : S3

Running on Docker? (yes/no) :

Additional context
Add any other context about the problem here.
Stacktrace
Add the stacktrace of the error.

@Amar1404 Sorry for delay here missing this one out. I will look into this and get back to you.

@ad1happy2go - Any updated on this.

@Amar1404 As discussed, Ideally withColumnRenamed should not be slow, I will benchmark number with and without sanitisation and get back to you.

@ad1happy2go - Any update on this?

@Amar1404 Never got a chance to try out this. I will prioritise it next week. Thanks.

hi @ad1happy2go  - Any updates on this


Private repos are visible in plan detail view and build detail view

At least these 2 views show private repo info when they shouldn't.

This appears to be working correctly since the access control refactoring.


Reports status of the integration commit after the build has finished

When using the branch updater feature of Bamboo, while the tests are run against an integrated branch, the commit is not pushed until the build is completed. This PR proposes pushing status to the merge commit resulting from a branch updater after the entire chain is finished.

@HackAttack Any thoughts on this? We've been using this code on our Bamboo instance for some time now. I'm happy to answer any questions or concerns.

Thanks for the ping @alindeman. I suspect this, #14, and #20 all touch the same code—will take a closer look this week.


Dev kubernetesconfiguration microsoft.kubernetes configuration stable 2021 03 01

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Please ensure to add changelog with this PR by answering the following questions.

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
During Ignite conference - around 02 March, 2021.
When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
As soon as the Swagger is merged, this week, if possible.
If it's an update to existing version,  please select SDKs of specific language and CLIs that require refresh after swagger is published.
Since this is a new api-version, we need SDKs generated

[x] SDK of .NET (need service team to ensure code readiness)
[x] SDK of Python
[x] SDK of Java
[x] SDK of Js
[x] SDK of Go
[x] PowerShell
[x] CLI
[ ] Terraform
[ ] No, no need to refresh for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[x] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
- Adding a new API version
Adding a new service

[x] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

@jianyexi, this PR is to create a new api-version for our upcoming GA release.  Compared to our most recent Public Preview api-version 2020-10-01-preview, there are no changes - except a Swagger Linting fix.  Please help us get this merged at the earliest possible.

@jianyexi, this PR is to create a new api-version for our upcoming GA release.  Compared to our most recent Public Preview api-version 2020-10-01-preview, there are no changes - except a Swagger Linting fix.  Please help us get this merged at the earliest possible.

@jianyexi Jianye Xi FTE, this PR is to create a new api-version for our upcoming GA release. Compared to our most recent Public Preview api-version 2020-10-01-preview, there are no changes - except a Swagger Linting fix. Please help us get this merged at the earliest possible.

Ok, needs ARM review first.

@jianyexi Jianye Xi FTE, this PR is to create a new api-version for our upcoming GA release. Compared to our most recent Public Preview api-version 2020-10-01-preview, there are no changes - except a Swagger Linting fix. Please help us get this merged at the earliest possible.

Ok, needs ARM review first.

@jianyexi, ARM team has signed off.  Can you please help get this merged?

@jianyexi, ARM team has signed off.  Can you please help get this merged?


fixed sentence case

Docs Build status updates of commit d4b5997:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/interactable-object.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


mixpanel__sessions throws LISTAGG limit error

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the issue
When running mixpanel__sessions model, redshift throws the following error:
21:45:36  Database Error in model mixpanel__sessions (models/mixpanel__sessions.sql)
21:45:36    Result size exceeds LISTAGG limit
21:45:36    DETAIL:
21:45:36      -----------------------------------------------
21:45:36      error:  Result size exceeds LISTAGG limit
21:45:36      code:      8001
21:45:36      context:   LISTAGG limit: 65535
21:45:36      query:     933154
21:45:36      location:  string_ops.cpp:116
21:45:36      process:   query3_246_933154 [pid=3755]
21:45:36      -----------------------------------------------

Relevant error log or model output
No response
Expected behavior
The model should run without error
dbt Project configurations
# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: "dwh"
version: "1.0.0"
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: "curative-dwh"

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

log-path:

target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"

on-run-start:
  # TODO: create script that runs on staging to make sure that the _0cp and _prod schemas are in sync with any new landing tables

  # when a new schema is created, fix permissions
  # NEED TO FIGURE OUT PERMISSIONS FOR THIS -->  "{{ update_permissions_for_lnd_schemas() }}"

on-run-end:
  # drop any models not in dbt project. only on staging and dev (not prod and ci; too risky)
  - "{{ drop_orphaned_relations(['base', 'curated', 'aggregated'], ['mixpanel', 'fivetran_log'], False if target.name in ['staging'] else True) }}"
  - "{{ re_data.save_test_history(results) if 'prod' == target.name }}"

vars:
  force_row_limit: -1 # if want to enforce a row limit, like we do for CI env

  mixpanel:
    mixpanel_database: dwh
    mixpanel_schema: lnd_mixpanel
    event_table: "{{ source('mixpanel', 'event') }}{{ limit_clause_by_env(10000) }}"
    date_range_start: "2022-03-30"

  greenhouse_database: dwh
  greenhouse_schema: lnd_greenhouse
  greenhouse_using_prospects: true
  greenhouse_using_eeoc: true
  greenhouse_using_app_history: false
  greenhouse_using_job_office: true
  greenhouse_using_job_department: true

  fivetran_log:
    fivetran_log_database: dwh
    fivetran_log_schema: lnd_fivetran
    fivetran_log_using_transformations: false # this will disable all transformation + trigger_table logic
    fivetran_log_using_triggers: false # this will disable only trigger_table logic
    log: "{{ source('fivetran_log', 'log') }}{{ limit_clause_by_env() }}"

models:
  +bind: false # Materialize all views as late-binding
  +post-hook:
    - "{{ redshift.compress_table(this.schema, this.table, drop_backup=False) if this.type == 'table' }}"
    - "{{ test_late_binding_view() }}"

  re_data:
    +enabled: "{{ 'prod' == target.name }}"
    +schema: base
    internal:
      +schema: base

  redshift:
    +bind: true
    +schema: base

  greenhouse_source:
    +schema: base

  mixpanel:
    +schema: base
    staging:
      +schema: base
      +docs:
        show: False

  fivetran_log:
    +schema: base
    staging:
      +schema: base
      +docs:
        show: False

  dwh:
    +materialized: view
    +re_data_monitored: true

    base:
      +schema: base

      app_public:
        staging:
          +re_data_monitored: false
          +materialized: ephemeral
          +docs:
            show: False

      revcycle:
        staging:
          +materialized: ephemeral
          +re_data_monitored: false

      ups:
        +enabled: False # busted for now
      safegraph:
        +enabled: False # need to setup
      rd:
        +enabled: False # need to setup

      twilio:
        twilio_call:
          +enabled: False
        twilio_role:
          +enabled: False
        twilio_role_permission:
          +enabled: False
        twilio_service:
          +enabled: False

    curated:
      +schema: curated
      core:
        cur_core_safegraph_visits_by_home_cbg:
          +enabled: False # need to setup

    aggregated:
      +schema: aggregated

sources:
  greenhouse_source:
    greenhouse:
      application_history:
        +enabled: False

  #dwh:
  #  landing:
  #    src_quadax:
  #      +re_data_monitored: true
  #    src_app_public:
  #      +re_data_monitored: false

seeds:
  +schema: base

tests:
  fivetran_log:
    +enabled: False

Package versions
  - package: dbt-labs/redshift
    version: 0.6.0

  - package: dbt-labs/dbt_utils
    version: 0.8.0

  - package: dbt-labs/dbt_external_tables
    version: 0.8.0

  - package: dbt-labs/codegen
    version: 0.5.0

  - package: fivetran/fivetran_utils
    version: 0.3.4

  - package: fivetran/fivetran_log
    version: 0.5.2

What database are you using dbt with?
redshift
dbt Version
1.0
Additional Context
No response
Are you willing to open a PR to help address this issue?

[ ] Yes.
[ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
[X] No.

@marclipoff-curative thanks so much for opening this issue. I am just starting to look into the error and noticed the LISTAGG issue is occurring within our mixpanel__sessions model at the following line.
https://github.com/fivetran/dbt_mixpanel/blob/7b05388f184b4b124d757d8c68ec5507db5d5e6e/models/mixpanel__sessions.sql#L115
The output of this event frequencies field will create a JSON object of the frequency of each event_type during this user session. As Redshift has a limit of 65535, it seems your sessions are having more frequency than the allotted amount. Before digging into the solution I wanted to ask a few quick questions:

Is this event_frequencies field important to you? You could analyze it further by exploding the JSON object, but curious if you find this field valuable for your current analysis. If you do not, then we can easily set a variable that will disable this field if you want.
If you find the event_frequencies field valuable, would it defeat the integrity of the field if we set a limit on the LISTAGG? For example, if we limited the LISTAGG to 50000 would you find the integrity of the field to be compromised since we are artificially stopping the event_type frequency?

For arguments sake, I actually ended up quickly removing this field if you wanted to see the package models succeed and you can evaluate the outcome and consider the importance of this field. You can leverage this version of the package by adding the below package configuration to your packages.yml
packages:
  - git: https://github.com/fivetran/dbt_mixpanel.git
    revision: hotfix/frequency-variable
    warn-unpinned: false

Let me know your thoughts!

I dont need it at the moment. So I'd be fine excluding it. When running that hotfix, I get:
16:09:04  Running with dbt=1.0.1
16:09:04  [WARNING]: Deprecated functionality
The `source-paths` config has been renamed to `model-paths`. Please update your
`dbt_project.yml` configuration to reflect this change.
16:09:04  [WARNING]: Deprecated functionality
The `data-paths` config has been renamed to `seed-paths`. Please update your
`dbt_project.yml` configuration to reflect this change.
16:09:05  [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- sources.dwh.landing.src_helpscount

16:09:05  Found 443 models, 505 tests, 0 snapshots, 4 analyses, 832 macros, 2 operations, 4 seed files, 633 sources, 0 exposures, 0 metrics
16:09:05
16:09:09  Concurrency: 10 threads (target='prod')
16:09:09
16:09:09  1 of 3 START view model base.stg_mixpanel__event_tmp............................ [RUN]
16:09:12  1 of 3 OK created view model base.stg_mixpanel__event_tmp....................... [CREATE VIEW in 3.27s]
16:09:13  2 of 3 START incremental model base.mixpanel__event............................. [RUN]
16:12:06  2 of 3 OK created incremental model base.mixpanel__event........................ [INSERT 0 2800820 in 172.16s]
16:12:06  3 of 3 START incremental model base.mixpanel__sessions.......................... [RUN]
16:12:06  3 of 3 ERROR creating incremental model base.mixpanel__sessions................. [ERROR in 0.81s]
16:12:07
16:12:07  Running 2 on-run-end hooks
16:12:07  Beginning drop of orphaned relations. dry_run=True
16:12:07  This command would be run: drop view if exists base.redshift_admin_running_queries cascade;
16:12:07  1 of 2 START hook: dwh.on-run-end.0............................................. [RUN]
16:12:07  1 of 2 OK hook: dwh.on-run-end.0................................................ [OK in 0.00s]
16:12:07  2 of 2 START hook: dwh.on-run-end.1............................................. [RUN]
16:12:07  2 of 2 OK hook: dwh.on-run-end.1................................................ [OK in 0.00s]
16:12:07
16:12:07
16:12:07  Finished running 1 view model, 2 incremental models, 2 hooks in 182.13s.
16:12:08
16:12:08  Completed with 1 error and 0 warnings:
16:12:08
16:12:08  Database Error in model mixpanel__sessions (models/mixpanel__sessions.sql)
16:12:08    syntax error at or near "("
16:12:08    LINE 128:     listagg((event_type || ': ' || number_of_events), ', ')
16:12:08                         ^
16:12:08
16:12:08  Done. PASS=2 WARN=0 ERROR=1 SKIP=0 TOTAL=3```

Sounds good, thanks @marclipoff-curative!
Were you able to run dbt clean before installing the new package? Also, would you be able to run dbt run --full-refresh to make sure we skip over any holdover incremental logic from past runs?

I think the problem is the stray ( . right?

Where do you see the stray (? The error is a bit strange since I completely removed the listagg function from the branch.

It compiles to this:
agg_event_types as (

    select
        session_id
        -- turn into json
        -- '{' ||
    listagg((event_type || ': ' || number_of_events), ', ')

 || '}' as event_frequencies

    from (

        select
            session_id,
            event_type,
            count(unique_event_id) as number_of_events

        from session_ids
        group by session_id, event_type

    ) as sub group by session_id
),

I think the fivetran_utils.string_agg macro creates a new line. So when commenting out using a --, the new line doesn't get commented out.

Actually, I was just messing around and think I found a way for users to leverage this field even if there are a few fields that exceed the limit. See my code below
https://github.com/fivetran/dbt_mixpanel/blob/3b7fcc10ead74c88cccddc2edb890a38c4aee15c/models/mixpanel__sessions.sql#L115-L122
Would you be able to test the new version of the branch as well and let me know if it succeeds for you? If it doesn't, you can try and adjust the new mixpanel__event_frequency_limit variable to be smaller than the default 50,000 I set (50,000 may be too high of a default haha).
vars:
   mixpanel__event_frequency_limit: 1000

Thanks! I will also limit the default to maybe 1000 to hopefully help with the string agg explosion as well

Closing this issue as PR #27 addressed the initial issue identified within this bug report.


Run functional tests on SG

[ ] CBLite 1.1 with sync_gateway 1.2
[ ] CBLite 1.2 with sync_gateway 1.1
[ ] CBLite 1.2 with sync_gateway 1.2
[ ] CBLite 1.2 (forestdb) with sync_gateway 1.2

I created all flavor of LiteServ Android
https://github.com/couchbase/couchbase-lite-android-liteserv

release/1.2.0-sqlite
release/1.2.0-sqlite-custom
release/1.2.0-sqlcipher
release/1.2.0-forestdb
release/1.2.0-forestdb-encryption

Notes:

Needs to uninstall application before switching storage type. They share same database name, encrypted database can not be opened by non-encryption support engine.
They does not requires submodule update.

hi, do I need to test release/1.1.1 or release/1.1.0 when you say sync_gateway 1.1?

I think release/1.1.1 is fine.

found only one new issue with the following configuration:
LiteServ Android 1.2.0-forestdb + SG release/1.1.1 https://github.com/couchbase/sync_gateway/issues/1509

Hi @andreibaranouski,
Is couchbase/sync_gateway#1509 Couchbase Lite Android issue? or SyncGateway issue?

#1509 is SG issue in release 1.1. it was fixed in 1.2 SG


Seal traits inside impl_ submodule

Off the back of https://github.com/PyO3/pyo3/pull/3897#discussion_r1504721475
Should we review the traits inside the impl_ submodule, check which ones can be sealed, and do that? It's a reasonable safety mechanism to prevent users from trying to implement those things.

I labelled this "Good First Issue", as this is an internal refactoring which would be great to have.
A "sealed" trait has a private supertrait, thus making it possible to implement outside the crate. All of the PyAnyMethods and similar traits are sealed, for example. Take a look at src/sealed.rs as the trait we use as the supertrait for most sealing.

Brilliant, thanks 🙏

@davidhewitt quick question, is there any particular reason something should be or should not be moved? I want to make sure I fully understand this.

I don't think we should be moving any code, this issue is about upgrading trait Foo to trait Foo: Sealed where it makes sense to do so for the traits inside the impl_ submodule.

I see, so those traits that could be used by the user need to be Sealed right? Thanks for clarifying it @davidhewitt

Yes, in principle many things inside impl_ are accessible by users but we never intend them to use them directly (just via the macros). The worry is there might be traits in there which we would prefer users can't implement at all, hence the sealing.

Thank you for your patience, I was occupied by EuroPython preparation in the last few weeks. I plan to pick this one up again this weekend or next week.

I think the next step along here would be finding traits which make use of the private_impl! helper and changing them all to be Sealed instead (and remove the old helper).


Remove pgp signing for pypi artifacts

Is your feature request related to a problem? Please describe
PyPi recently announced that they will be Removing PGP from PyPI. See https://blog.pypi.org/posts/2023-05-23-removing-pgp/
Even though the upload is allowed it will be silently ignored.
Describe the solution you'd like
Remove https://github.com/opensearch-project/opensearch-build-libraries/blob/main/vars/publishToPyPi.groovy#L19-L23
Describe alternatives you've considered
Continue publishing but it will throw an error then the support is completely removed.
Additional context
No response

[Untriage]
Yes we can remove the .asc signing after confirming that if the uploaded tar package with twine does have some signing handled by PyPi.

Maybe remove PyPi library completely https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/

@gaiksaya Future implementation will be
https://docs.pypi.org/trusted-publishers/adding-a-publisher/
https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-pypi
as you have mentioned in https://github.com/opensearch-project/opensearch-build/issues/3822

Since Pypi no longer requires PGP signing, we plan to move the Pypi publishing from Jenkins to GHA.
According to my research, here are some steps:

Added existing projects in our account to trusted publisher. https://docs.pypi.org/trusted-publishers/adding-a-publisher/
Drafted GHA workflow yml with the same name we added in the step above.
Use pypa/pypi-publish from https://github.com/marketplace/actions/pypi-publish to configure publishing.

This task is completed. We have added onboarding doc here.

We need to create a campaign for all existing pypi artifacts. Please keep this issue open until we have that.
We might also need to depreacte the library from the vars folder for upcoming versions.
Thanks!

We need to create a campaign for all existing pypi artifacts. Please keep this issue open until we have that.
We might also need to deprecate the library from the vars folder for upcoming versions. Thanks!

I have created issue here in benchmark repo. https://github.com/opensearch-project/opensearch-benchmark/issues/451
I think that's the only active pypi artifact we released using our jenkins library.

I see few more: https://github.com/search?q=org%3Aopensearch-project+publishToPyPi&type=code
Maybe list down in this issue what are migrated to new process and remaining ones to be migrated.

I see few more: https://github.com/search?q=org%3Aopensearch-project+publishToPyPi&type=code Maybe list down in this issue what are migrated to new process and remaining ones to be migrated.

What is it? I don't see any repo from your link that needs us to migrate.

If we have already migrated then we need to off-board all the repos. Remove webhooks to jenkins and related jenkins file from the repos.

If we have already migrated then we need to off-board all the repos. Remove webhooks to jenkins and related jenkins file from the repos.

I think we should leave it to the repo owner. Neither of the team has released yet with the new onboarding GHA. We have done steps to help them onboard and they could remove it in the future since they won't be using it anymore.

We will open new issue to track the lib removal and off-boarding existing components.


Test the application gateway- typographical error in step 3

Test the application gateway;
Change the URL to http://:8080/video/test.htm, substituting  with your IP address, and you should see something like the following example:
should replace by
Change the URL to http://:8080/images/test.htm, substituting  with your IP address, and you should see something like the following example:

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 36e72b72-f20c-8775-5026-caef1d1d2a01
Version Independent ID: 62f50079-7903-91f2-f2c3-dc617241ef32
Content: Create an application gateway with URL path-based routing rules - Azure portal | Microsoft Docs
Content Source: articles/application-gateway/application-gateway-create-url-route-portal.md
Service: application-gateway
GitHub Login: @davidmu1
Microsoft Alias: davidmu

@sanjayincloud Thank you very much for your feedback. We will investigate your inquiry and get back with you.

@sanjayincloud, thanks for catching this. The article has been updated:
https://docs.microsoft.com/en-us/azure/application-gateway/application-gateway-create-url-route-portal
#please-close

@sanjayincloud We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


[TVMC][MicroTVM] tvmc micro project_dir arg does not accept relative path

Expected behavior
Expect tvmc micro create-project to run without error.
Actual behavior
The following error occured on the Project API server side:
 calling method generate_project: JSON-RPC error # -32000: calling method generate_project
Traceback (most recent call last):
  File "/home/mhessar/tvm/python/tvm/micro/project_api/server.py", line 481, in serve_one_request  # <--- Outermost server-side stack frame
    self._dispatch_request(request)
  File "/home/mhessar/tvm/python/tvm/micro/project_api/server.py", line 593, in _dispatch_request
    return_value = dispatch_method(**params)
  File "/home/mhessar/tvm/python/tvm/micro/project_api/server.py", line 625, in _dispatch_generate_project
    options,
  File "/home/mhessar/tvm/build/microtvm_template_projects/arduino/microtvm_api_server.py", line 338, in generate_project
    project_dir.mkdir()
  File "/usr/lib/python3.6/pathlib.py", line 1248, in mkdir
    self._accessor.mkdir(self, mode)
  File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: 'tvmc_relative_path_test/project'

Steps to reproduce
poetry run tvmc micro create-project ./tvmc_relative_path_test ./output_dir/model.tar arduino --project-option "project_type=host_driven"

cc @gromero @guberti

fixed by https://github.com/apache/tvm/pull/9663


Table allignment issue in NVSK

Table allignment is not proper
NAS (grade and subject performance)

Issue is Fixed


General: Include more information in course archive

Checklist
General

[ ] I tested all changes and their related features with all corresponding user types on a test server.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
[x] I chose a title conforming to the naming conventions for pull requests.

Server

[x] Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
[x] I followed the coding and design guidelines.
[x] I added multiple integration tests (Spring) related to the features (with a high test coverage).
[x] I documented the Java code using JavaDoc style.

Motivation and Context

Course archives do not contain exercise details or the problem statement for all exercise types. This PR adds this information to course archives.
Description
Refactored the SubmissionExportService. Fixed the exports of File Upload submissions. Introduced a new ExerciseWithSubmissionsExportService that exports a JSON file with information about the exercise, a markdown file with the problem statement and all submissions or student repos for an exercise.  For programming exercise, we also export template, solution, test, and aux repositories.
Steps for Testing

Prerequisites:

1 Instructor
2 Students
1 Programming Exercise with Complaints enabled

Log in to Artemis
Navigate to Course Administration
Select a course that has an end date in the past or set the end date of a course to a value in the past.
Click on archive
Once finished, download the archive and verify its content.
Every exercise should have its own directory with the problem statement as markdown file, the exercise details json file and all submisisons.

Review Progress

Performance Review

[x] I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance
[x] I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

[ ] Code Review 1
[ ] Code Review 2

Manual Tests

[ ] Test 1
[ ] Test 2

Exam Mode Test

[ ] Test 1
[ ] Test 2

Test Coverage

Screenshots

Tested on TS3 and it works as expected.


added the neccessary files

Description 📜
added the necessary files in the required folder

Type of change 📝

[x] Documentation (Content Creation in the form of codes or tutorials)

Domain of Contribution 📊

[x] JavaScript

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

@Prateek-8168 Please ensure your PR follows the JavaScript contribution guide: https://github.com/girlscript/winter-of-contributing/blob/Javascript/JavaScript/CONTRIBUTING.md

Hey Prateek! Are you working on this PR?


Start to safely partially initialize slices

Currently we are using MaybeUninit::uninit().assume_init() is an
undefined way. This patch uses MaybeUninit::uninit().assume_init() to
create arrays of MaybeUninit which is defined behavior. Slices of
these arrays have their elements set and the slices of initialized
elements are transmuted to a slice of T.
One instance of full array initialization is replaced and one instance
of undefined behaviour is replaced.

yeah, clippy's suggestion works.


For a 1.3 release

#968 #975 #973 #969

For the release notes:

(Performance enhancement) Remove type instability for predict(mach::Machine, ...) in the easy and typical case that mach does not wrap a Symbol model (#969)

(New feature) Give evaluate and evaluate! the option compact=true, to return  a CompactPerformanceEvaluation object with minimal memory footprint (#973)

(New feature) Add an InSample() resampling strategy that trains and tests on the same data (whatever is specified by rows, or all supplied data) (#975)

(Display improvement) Split the table displayed as part of an PerformanceEvaluation object over two tables, if needed, to deal with overly wide tables (#973)


Move button in UI changes to "Saving" and stays spinning when a namespace is moved from a project without resource quota to a project with resource quota

Rancher versions:
rancher/server or rancher/rancher: Master
Steps to Reproduce:

Create a project "P1" without any resource quota assigned and create a namespace "N1" in P1
Create a project P2 with resource quota assigned.
Move N1 to project P2

An error pops up that move is not allowed as expected. But the move button changes to "Saving" and stays spinning as below.

Also, can the error displayed in red on the side be changed to something better ?

Version - 2.1 master 9/17
Saving fixed.


New API proposal-Shutdown Service Status

This API allows users to obtain the current shutdown status of a certain phone number, including Voice/SMS in shutdown, Voice/SMS out shutdown, and data service shutdown.
Anti fraud: When receiving SMS or phone calls from a suspended number, it may be a disguised fraudulent number, which can provide security risk alerts.
Customer service support: If a user reports that they are unable to receive calls or text messages, customer service personnel can determine whether the problem is caused by the phone voice function being shut down by checking the phone's status.
This is from GSMA drop4 priority list
PR is here:32

Let me close this issue as I have migrated the existing one to keep the record of the conversation, thank you @chinaunicomyangfan


Tests for Hexa Demo App Need Updating

Current test file is disabled and needs re-writing.

A new demo_test.go was added.


prepare for CTAN release

This is a PR for CTAN release preparation.
CTAN has a strict check on package release.

Upload Addendum.
Package Structure.

It may take some time for the package to pass the release.  Meanwhile:

Check copyright licenses. I believe the fduthesis is of the best similarity on logo copyright.
Check installation.

Remember the namespace of our package is sjtubeamer, which is not the name of this repository. Because:

Let us remind you that there are operating systems unable to distinguish between myfile and MyFile . That's why we check for “identical filenames” after converting everything to “lowercase”.

Local Installation check complete. I will add a README in the src/ directory to make it included in the package.

First attempt on CTAN release ...

忘了

I am searching the information on vi/ folder inclusion. It is dangerous to include such a directory since it could make it to the MiKTeX distribution. See hitszbeamer for a failed output.
I will try to solve it through TDS packaging standard.

We could drop support for MiKTeX for now :) We could resolve this in next minor version.

I am not sure about that... I could be solved by some way... I guess...
I will try to figure it out.

It seems fibeamer is successfully built. I have to believe *-ctan.zip with its tds.zip will work, at least for now. It seems to be the previous beamer is not correctly packaged.

First attempt on CTAN release has been made. Notice the email.

Technical error occurred. Internal distribution will be first deployed.

#77 will try to solve the technical errors.


"dep init" fail with error "No versions of k8s.io/client-go met constraints"

(Related issue: #176,  but it is not the same.)
Repro:
$ export GOPATH=~/workspace/client-go-examples

$ mkdir -p $GOPATH/src

$ ( for this step, copy contents of examples/ directory in the repo to ./src as that's what INSTALL.md suggested)

$ cd  $GOPATH/src/examples

$ ls
in-cluster  out-of-cluster  third-party-resources  workqueue

$ dep init
Cached github.com/golang/glog
Cached k8s.io/apimachinery
Cached k8s.io/client-go
No versions of k8s.io/client-go met constraints:
	v2.0.0: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)
	v1.5.1: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/api/v1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/apis/extensions/v1beta1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/rest is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/cache is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/clientcmd is missing. (Package is required by (root).)
	v1.5.0: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/api/v1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/apis/extensions/v1beta1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/rest is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/cache is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/clientcmd is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes is missing. (Package is required by (root).)
	v1.4.0: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/tools/cache is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/clientcmd is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/api/v1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/apis/extensions/v1beta1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/rest is missing. (Package is required by (root).)
	v3.0.0-beta.0: Could not introduce<EMAIL_ADDRESS>as it requires package github.com/emicklei/go-restful/swagger from github.com/emicklei/go-restful, but in version 2.2.0 that package is missing.
	v2.0.0-alpha.1: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)
	v2.0.0-alpha.0: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)
	master: Could not introduce k8s.io/client-go@master, as it requires package github.com/emicklei/go-restful/swagger from github.com/emicklei/go-restful, but in version 2.2.0 that package is missing.
	release-1.4: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/api/v1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/apis/extensions/v1beta1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/rest is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/cache is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/clientcmd is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes is missing. (Package is required by (root).)
	release-1.5: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/pkg/api/v1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/apis/extensions/v1beta1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/rest is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/cache is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/clientcmd is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)
	release-2.0: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)
	release-3.0: Could not introduce<EMAIL_ADDRESS>as it requires package github.com/emicklei/go-restful/swagger from github.com/emicklei/go-restful, but in version 2.2.0 that package is missing.
	revert-14-1.5: Could not introduce<EMAIL_ADDRESS>due to multiple problematic subpackages:
	Subpackage k8s.io/client-go/kubernetes/scheme is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/api/v1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/pkg/apis/extensions/v1beta1 is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/rest is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/cache is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/tools/clientcmd is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/util/workqueue is missing. (Package is required by (root).)	Subpackage k8s.io/client-go/kubernetes is missing. (Package is required by (root).)

The line "ImportPath": "github.com/emicklei/go-restful/swagger", in Godeps.json references a non-existant package. It is causing a problem for me as well, trying to save my project dependencies with godep.

It was removed in emicklei/go-restful@24f362968e4ac4f9f0a7f6c74ac4d66369716e16. I fixed it by checking out the commit before that (tag 1.0.0) in $GOPATH/github.com/emicklei/go-restful

Had the same issue when including k8s.io/client-go in a custom project. Solved it by adding the following lines to the Gopkg.toml:
[[override]]
  name = "github.com/emicklei/go-restful"
  revision = "ad3e7d5a0a11fbbead57cc9353720a60a0a2793f"


Update CRT submodules to latest releases

Description of change
Pulling in the latest changes.
9e11ad8 Update CRT submodules to latest releases
Submodule mountpoint-s3-crt-sys/crt/aws-c-auth 71bad38..50f3e0d:
  > Add an Option to Disable v2 to v1 Fallback (#218)
Submodule mountpoint-s3-crt-sys/crt/aws-c-common fefbf4b..80f21b3:
  > Fixed directory creation to ignore umask, and allow cross process loc… (#1078)
Submodule mountpoint-s3-crt-sys/crt/aws-c-io c9cb777..df64f57:
  > Use 500ms rather than 25ms for the exponential backoff defaults. (#612)
  > Remove null check for aws_pkcs11_lib_login_user (#611)
Submodule mountpoint-s3-crt-sys/crt/aws-c-s3 83008e5..cc6ba34:
  > Report S3 operation name of specific request that failed. (#377)
  > Auto - Update S3 Ruleset & Partition (#376)
  > Mem limiter (#368)
  > fix stale bot permissions (#371)
Submodule mountpoint-s3-crt-sys/crt/aws-lc a8d06de..80f3f33:
  > Bump release version number string to 1.17.4 (#1319)
  > Improve Codecov reporting (#1313)
  > Merge pull request #1314 from torben-hansen/aws-lc-s2n-bignum-update-2023-11-19
  > Merge pull request #1308 from torben-hansen/aws-lc-s2n-bignum-update-2023-11-15
  > Add BNAssertTest (#1267)
  > Silence warning on CMP0116 (#1284)
  > Use OPENSSL_DEPRECATED (#1285)
  > Add integration ci for Monit (#1286)
  > Include Codecov in CI (#1307)
  > Fix bn_assert_fits_in_bytes for big-endian (#1258)
  > Fix-up curve25519 code to prepare for s2n-bignum (#1306)
  > Merge pull request #1302 from samuel40791765/upstream-merge-2023-11-10
  > Add check for sk_X509_push (#1304)
  > Refactor ED25519_verify into hw and nohw backend (#1305)
  > Add back OCSP integration test executing (#1303)
  > Refactor ED25519_sign into hw and nohw backend (#1276)
  > Add HybridKeyShare support for SecP256r1Kyber768Draft00 and X25519Kyber768Draft00 (#1201)

Relevant issues: N/A
Does this change impact existing behavior?
This change inherits a change of default when constructing a CRT client using mountpoint-s3-crt, however mountpoint-s3-client already sets its own defaults and is unaffected.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

We should update our bindings based on https://github.com/awslabs/aws-c-s3/commit/cc6ba346b55ef012f7131d98dcc68e16acc16d95.


Fixes #844 -- docs deployment

@dflanigan
@ssorenso
What issues does this address?
Fixes #844
What's this change do?
Updates how the variable that determines the publication path for docs is set.
This should prevent us from having further issues with deployment.
Where should the reviewer start?
Any background context?

Can one of the admins verify this patch?

Can one of the admins verify this patch?


✨ Adds 'shell' rule for checking cli output via regexp

This adds a new shell rule.  For example:
{
  "git email": [{
    "rule": "shell",
    "command": "git config user.email",
    "match"<EMAIL_ADDRESS>  }]
}

I opted out of the expression evaluation since bc can do some (or all?) of it.
I also left off the exit code stuff.  I'm not sure if it's best in here or a new exit code rule?  Either way, I don't have an immediate need for it.  This matching stuff will probably fulfill most of the needs for now.
Closes #133

AppVeyor is failing because of the upstream gluegun issue.

Adding this new rule to the report functionality!


adding stastistical power analysis description to raw data

Information about eventual statistical power analysis

could you go into a bit more detail and perhaps provide an example of what you would think such a field would look like?
Originally posted by @sappelhoff in https://github.com/bids-standard/bids-specification/issues/552#issuecomment-667657999
This is pretty much the info you have about this in the spreadhseet COBIDAS for MRI

info
description or example

Source of predicted effect size
previous literature with citation; pilot data with description

Significance level
uncorrected alpha 0.05 for an ROI

Target power

Any other parameters set (e.g. for spatial methods a brain volume and smoothness may be needed to be specified)

As far as I know the info in the EEG / MEG COBIDAS is lighter than this but I would prefer to make them consistent across modalities.

closing this for now
not so sure anymore this is within the scope of BIDS


Update SinkOptions.cs

Correct Spelling error within File from tabke to table on property AutoCreateSqlTable.

Thank you very much for the correction! 🙇


DOC: minor stylistic revision, undo changes

Minor stylistic changes that had to be undone from gh-11157.
@rgommers Thanks so much for reviewing the massive changes so carefully!

Merged, thanks @mkg33


Pluggable page types

Pages have a few different types now, that maybe could inform a pluggable page builder API.
For example:

https://giuseppegurgone.com/xm/
isomorphic uland/uhtml?

Nah


'String search' to include H_INVOKESTATIC Handle args

'String search' to include H_INVOKESTATIC Handle args[]
Be able to search strings in the args[] passed to the H_INVOKESTATIC handle used in InvokeDynamic
Is it possible? thanks :)

Will be in the next version, it's quire simple to add.


Problems installing CM01 - Install SQL Server 2019

I have updated the following files
ConfigurationFile.ini - line 147 with SQLSYSADMINACCOUNTS="TIMBO\Administrator" "BUILTIN\Administrators"
Install-HYDSQLServer2019.ps1 - line 133 with  $OriginalSQLSYSADMINACCOUNTS = "SQLSYSADMINACCOUNTS="TIMBO\Administrator" "BUILTIN\Administrators""
Note that CU15 (SQLServer2019-KB5008996-x64.exe) is in the Updates folder
But there seems to be an error displayed as indicated in attachment
Any help, much appriechated
Regards
Tim

I think I have fixed this - it is because the $tsenv is not know at this point . Adding the $tsenv = New-Object -COMobject Microsoft.SMS.TSEnvironment line before it - fixes it (not elegant - but works)

Thanks for the info, I have updated the installation script with that change.

That's one of the default packages. Verify that the DP role is happy, and then just run an Update Distribution Point on that package. I've seen it sometimes not being distributed by default.


Adds HoodieGlobalBloomIndex

WHAT

feature(HoodieGlobalBloomIndex): adds a new type of bloom index to allow global record key lookup

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

In general, very concise PR @leletan, just what was needed!

@vinothchandar Thanks for the advices, will make change accordingly. Not yet validated with data set, will do it.

A quick small data set test indicates the cross partition deduping is not working, will dig deeper.

One thing I observed is that there is still a parquet file created for a microbatch with records only containing dups to the records saved in table, even for the same partition dups.

@vinothchandar wondering if there is any use case I may run some dataset to test this PR, other than cross partition deduping.

Thats the main use-case.. Can't think of anything else top of my head.
but there can be several corner cases like

ensuring the partitionpath supplied in the RDD, has no bearing - i.e all keys checked against all files..
generate input such that partition paths in input RDD are a subset of total partitions in dataset (HoodieTestDataGenerator may have them to be same by default)..

@leletan @n3nash I think I may know why global de-duping may not be working as expected
/**
   * For each incoming record, produce N output records, 1 each for each file against which the
   * record's key needs to be checked. For datasets, where the keys have a definite insert order
   * (e.g: timestamp as prefix), the number of files to be compared gets cut down a lot from range
   * pruning.
   */
  // sub-partition to ensure the records can be looked up against files & also prune
  // file<=>record comparisons based on recordKey
  // ranges in the index info.
  @VisibleForTesting
  JavaPairRDD<String, Tuple2<String, HoodieKey>> explodeRecordRDDWithFileComparisons(

this method still only joins input records using the partitionpath supplied in the HoodieKey, this needs to be overridden and fixed as well..
@n3nash are you able to take a pass here and provide a path for @leletan ? Also please take a look at usages of HoodieIndex::isGlobal and see if they will still be okay..

@vinothchandar Yes, I can drive this. You are right, JavaPairRDD<String, Tuple2<String, HoodieKey>> explodeRecordRDDWithFileComparisons(, this needs to be overloaded for 2 reasons 1) To be able to autoComputeParallelism correctly and 2) For finding the correct matching files (which in this case should be all).
@leletan Please override that method as well. Also, I took a closer look at HoodieIndex::isGlobal, I think it's best to return ** false ** here. There are couple of places in the code which has assumptions about what isGlobal=true means and this implementation does not either satisfy those or does not apply to that assumption. There should be no side-effect of setting it to false.
@vinothchandar We need to remove those assumptions from the code (they are specifically there because of HbaseIndex), that would make isGlobal understanding clean.

@leletan Just checking back here, any progress/updates on this ?

@n3nash Was still in the progress of making the end-to-end data test work.
Firstly was stuck by this and found a work around in that ticket.
Then I was kind of stuck but just found a way to do a dedup later record here to make sure the cross partition dedup in this PR will keep the record in the old partition in my end-to-end testing.
And next step I am going to continue this work based on your input.
But eventually my test will be based on changes on top of this branch since we are using spark 2.3 in our production.
Wondering if you guys have any suggestions for end-to-end testing for this PR or if the HoodieJavaApp is good enough

@leletan thanks for the update. Definitely run the PR against HoodieJavaApp, also if you can try to run the PR against some dataset internally, that will help vet the PR more. May be run the dataset against the latest release (without your PR) and then with your PR, you can validate if your dataset behaves correctly in both scenarios; that's a good way to vet your code.
We don't have an exhaustive set of end to end testing as such (as we discussed f2f) but unit tests should capture most of the quirks.

Made some new modification based on comments and also added some unit tests. Will continue to do some real data testing later.

Ran data with HoodieJavaApp - seems working fine as expected. Now will put this into real spark ETL job for a test

@leletan sounds great, let me know how it goes.

nicee!

I had some changes based on discussion in #441, please let me know if it make sense to add that code into this PR or better to make a separate PR for that.

Tested this with our real spark ETL job along with some changes I made for #441, things are working fine.

@leletan Please open a separate PR for changes made for changes talked in the #441. I'll take another pass at this PR later tonight.

@leletan Left a few comments, rest looks ok.

Made change according to the comments.
If all things look fine, let me know if a final rebase is needed to have cleaner commit log.

@leletan Left a few comments on the TestUtils usage, once that is done please squash your commits to a single commit.

@leletan Please squash the commits and let me know when the diff is ready. I understand the cyclic dependency on HoodieTestUtils so let's avoid that. Please see if there are utils that you can reuse for creating datafiles etc (instead of creating them in the code as you have right now). We can get this diff landed soon.

Good point will do

Done

@leletan Left 2 minor comments after which this is good to merge.
@vinothchandar FYI

Done. Also rebased and squashed the commits

@vinothchandar LGTM


pip package

Could you make a pip package? :)

I've created one unofficially at https://pypi.org/project/nisqa/2.0/ and the files required to build it are in #47

Is there a python package for NISQA. In short, what would be the best way if we want to use the NISQA model during runtime? I would like something like,
wav = librosa.load(wav_path)
mos = NISQA(wav)

Sorry for the wrong link I posted previously (there was a packaging error so the version number is now 2.0.post2). You can just install the nisqa package using pip. You can use the same functions as shown in the run_train.py and run_evaluate.py files.

Is there any way in which NISQA can read a loaded wav file during runtime instead of reading .wav file from disk?


libcypher-parser allows clause sequences prohibited by the grammar

Cypher has some restrictions in its grammar file regarding clause sequences. For example, in a single-part query, a reading clause cannot follow a writing clause, and no clause can follow RETURN.
libcypher-parser allows these constructions, however, constructing ASTs for queries such as:
CREATE (a) MATCH (a)
RETURN 1 DELETE a
RETURN 1 RETURN 1

Shouldn't these constructions trigger parser errors?

I'd say they are semantic errors rather than parse errors. Adding a semantic checker to libcypher-parser has been a long held desire.


Issue with choosing color temperature colors on gledopto 1ID controller...

So if I'd go and choose either warm or cold white, homekit would override it with (I guess not random) color - therefore I can't really create automations with cold/warm white on RGBCCT strips, does anyone have an idea why that might be happening?

Because Home doesn't support all of HomeKit.  Use Eve to create the HomeKit scene instead.

But it does work with regular Hue lights?

Home doesn't support both Hue / Saturarion and Color Temperature on the same Lightbulb service.  So when you think you select a colour temperature using the wheel, Home actually sends Hue and Saturation to the accessory (in this case: Homebridge Hue).
This is no issue for the older "regular" gamut-B Hue lights, that only support three channels, RGB, and create colour temperatures using these.  They've optimised the RBG LEDs on these lights, so they produce proper colour temperatures, at the expense of full blue and green.
It is an issue for the newer "regular" gamut-C Hue lights and for the GLEDOPTO, which support five channels, RGB/WW/CW.  Sending Hue and Saturation drives the RGB channels (through setting xy), whereas sending Color Temperature drives the WW/CW channels (through setting ct).
I worked around this issue by using an old custom Color Temperature characteristic (from before Apple defined the standard characteristic).  Eve still recognises it, and Home happily ignores it.

 I have more than 20 gledopto controllers that I want to use :/
As workaround I thought about exposing these Phillips Hue sccenes to homekit as switches and toggle colors that way, but I can't get scenes to show in homekit, any idea why?

You can use HomeKit scenes alright, you just need to create them in Eve.  You can still recall them from Home and Siri.
To expose scenes on the Hue bridge or deCONZ, set groups, group0, and scenes to true in config.json.  Or whitelist the groups whose scenes you want to expose.  The scenes are exposed as additional Switch services to the accessory for the group.

Hey man! Thank you, I've been messing with this for a few days now, and kindof figured how I will use it...
One last question, would 2ID controller give me an option to control temperature with that 2nd light which would be only white, or would it do the same thing? Thanks!

I never laid my hands on a 2ID controller, but I think it works like the old RGBW controllers: it exposes two endpoints, one Color Light to drive the RGB channels, and one Color Temperature light to drive the CW/WW channels.  I think you can drive both channel groups independently, and at the same time, whereas the 1ID drives either RGB or CW/WW.
Homebridge Hue should exposes the 2ID as two separate Lightbulb services in a single accessory, one with Hue and Saturation, the other with the Apple standard Color Temperature.

Yeah, thats right, but I was wondering if that other light would control actual color temperature of strip or would it do same thing as 1ID and try to mimic colors with RGB part of the strip... I guess I'll order one and give it a try :D

The 1ID light doesn’t mimic anything.  Apple’s Home app does.  As it will see a Lightbulb with only Color Temperature, it won’t display the colour wheel, and send Color Temperature changes, which Homebridge Hue translates to ct.


Update messages_pl_PL.properties

Adjusting the translation to the new version

Hi MarcO-79,
We are updating Stirling-PDF's license to MPL 2.0 (Mozilla Public License Version 2.0) to continue fostering our open-source commitment while introducing more flexible usage of the project.
What We Need From You:
In order to transition to the MPL 2.0 license, we need the explicit consent of all our contributors.

If you're comfortable relicensing your contributions for this specific PR under MPL 2.0, please respond with:
I, MarcO-79, grant permission to relicense my contributions from this PR to Stirling-PDF under the Mozilla Public License 2.0.
If you'd prefer to give blanket permission for all of your contributions to Stirling-PDF (be it through pull requests, commits, or any other form of contribution) rather than on a PR-by-PR basis, please respond with:
I, MarcO-79, grant permission to relicense all my contributions to Stirling-PDF under the Mozilla Public License 2.0.

What does this do and why?
As your old code was contributed under a different license it legally cannot move to a new license without your permission, and since we dont want to lose your code we wanted to reach out. Remember this project is remaining open-source!.
Deadline:
If we dont receive a response, or if you decline, unfortunately, we will have to remove your contributions from Stirling-PDF before the license change, which is scheduled  in coming months.
Thank You!
Your contributions are invaluable, and we hope to continue having them in Stirling-PDF under the new license. Feel free to ask if you have any questions or concerns about this change.
Best,
Frooodle
Stirling-PDF Maintainer

Hi MarcO-79,
We are updating Stirling-PDF's license to MPL 2.0 (Mozilla Public License Version 2.0) to continue fostering our open-source commitment while introducing more flexible usage of the project.
What We Need From You:
In order to transition to the MPL 2.0 license, we need the explicit consent of all our contributors.

If you're comfortable relicensing your contributions for this specific PR under MPL 2.0, please respond with:
I, MarcO-79, grant permission to relicense my contributions from this PR to Stirling-PDF under the Mozilla Public License 2.0.
If you'd prefer to give blanket permission for all of your contributions to Stirling-PDF (be it through pull requests, commits, or any other form of contribution) rather than on a PR-by-PR basis, please respond with:
I, MarcO-79, grant permission to relicense all my contributions to Stirling-PDF under the Mozilla Public License 2.0.

What does this do and why?
As your old code was contributed under a different license it legally cannot move to a new license without your permission, and since we dont want to lose your code we wanted to reach out. Remember this project is remaining open-source!.
Deadline:
If we dont receive a response, or if you decline, unfortunately, we will have to remove your contributions from Stirling-PDF before the license change, which is scheduled  in coming months.
Thank You!
Your contributions are invaluable, and we hope to continue having them in Stirling-PDF under the new license. Feel free to ask if you have any questions or concerns about this change.
Best,
Frooodle
Stirling-PDF Maintainer


Build issues on Windows

I receive a build issue with OCICL on Windows that I do not see on QuickLisp, it maybe that it does not try to recompile swank  bankend that I assume is already compiled and part of sbcl
Maybe you have an idea?
; file: C:/Users/david/projects/rtest/systems/slime-2.30/swank/sbcl.lisp
; in: DEFUN INPUT-READY-P
;     (SB-BSD-SOCKETS-INTERNAL::FD->HANDLE (SB-SYS:FD-STREAM-FD STREAM))
;
; caught WARNING:
;   The function SB-BSD-SOCKETS-INTERNAL::FD->HANDLE has been deprecated as of SBCL version 1.2.10.
;
;   In future SBCL versions SB-BSD-SOCKETS-INTERNAL::FD->HANDLE will signal a
;   runtime error.
;   See also:
;     The SBCL Manual, Node "Deprecation Conditions"
; in: DEFVAR SHEBANG-READTABLE
;     (LAMBDA (STREAM CHAR) (VALUES))
;
; caught STYLE-WARNING:
;   The variable STREAM is defined but never used.
;
; caught STYLE-WARNING:
;   The variable CHAR is defined but never used.
; wrote C:/Users/david/AppData/Local/cache/common-lisp/sbcl-2.4.6-win-x64/C/Users/david/projects/rtest/systems/slime-2.30/swank/sbcl-tmpB7NPDQCW.fasl
; compilation finished in 0:00:01.479
debugger invoked on a UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread
#<THREAD tid=28040 "main thread" RUNNING {1000110093}>:
COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "swank" "backend" "sbcl">
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [RETRY                        ] Retry
compiling #<CL-SOURCE-FILE "swank" "backend" "sbcl">.
1: [ACCEPT                       ] Continue, treating
compiling #<CL-SOURCE-FILE "swank" "backend" "sbcl">
as having been successful.
2:                                 Retry ASDF operation.
3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the
configuration.
4:                                 Retry ASDF operation.
5:                                 Retry ASDF operation after resetting the
configuration.
6: [CONTINUE                     ] Ignore runtime option --eval "(asdf:load-system :clog/tools)".
7: [ABORT                        ] Skip rest of --eval and --load options.
8:                                 Skip to toplevel READ/EVAL/PRINT loop.
9: [EXIT                         ] Exit SBCL (calling #'EXIT, killing the process).
(UIOP/LISP-BUILD:CHECK-LISP-COMPILE-RESULTS NIL T T "~/asdf-action::format-action/" ((#<ASDF/LISP-ACTION:COMPILE-OP > . #<ASDF/LISP-ACTION:CL-SOURCE-FILE "swank" "backend" "sbcl">)))
error finding frame source: Bogus form-number: the source file has probably
changed too much to cope with.
source: NIL
0]

The following needed to allow swank to compile on windows
(setf asdf:compile-file-failure-behaviour :warn)
Is there a way to selectively turn such a thing on for a certain package?

Setting up OCICL
https://www.youtube.com/watch?v=CIDo8vaOy1c
First project with OCICL
https://www.youtube.com/watch?v=IZqHuYw9jRI


Close idle TCP connections

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
For changelog. Remove if this is non-significant change.
Category (leave one):

Improvement

Short description (up to few sentences):
Close idle TCP connections (with one hour timeout by default). This is especially important for large clusters with multiple distributed tables on every server, because every server can possibly keep a connection pool to every other server, and after peak query concurrency, connections will stall. This fixes #5879

How to test:
milovidov@milovidov-Latitude-E7470:~/work/ClickHouse/dbms/programs/server$ cat users.d/idle_connections.xml
<yandex><profiles><default>
    <poll_interval>1</poll_interval>
    <idle_connection_timeout>2</idle_connection_timeout>
</default></profiles></yandex>

Note: I'm not sure that we will add integration test.

Does server send FIN packet to the client, or connection is closed forcefully? If former, is latter possible?

I don't remember.


Pressing enter in the console does nothing

From @lynnewu
copied from https://github.com/NuGet/NuGetGallery/issues/2516
I've installed NuGet (Windows 7, VS 2013) and it seems to install correctly, but when I open the Package Manager Console and enter a command, nothing happens.
The cursor moves down to the next (empty) line, and if I keep pressing  it moves to the next line, but nothing ever happens.

This issue was moved to NuGet/Home#1329


Provide a dummy token module for tests/emulation

It would be nice to ship a token module that doesn't actually need to communicate with Google.  This would be really helpful in a test environment, but also when interfacing with the emulator, since it doesn't validate tokens.

Would the solutions discussed for SQS also be valid here?

Btw, we preferred to go with a MFArgs tuple instead of a module because the contract is simple, so there is no reason to force the overhead of defining a module. The token generator option is also easier to handler during testing. :)

Would the solutions discussed for SQS also be valid here?

Yes, this is totally valid here, too.

Btw, we preferred to go with a MFArgs tuple instead of a module because the contract is simple, so there is no reason to force the overhead of defining a module. The token generator option is also easier to handler during testing. :)

Agreed. Would you expect the call to the :token_generator would return just the token, or a named tuple?
For Goth, the default MFArgs tuple would look like this:
{Goth.Token, :for_scope, ["https://www.googleapis.com/auth/pubsub"]}

and it's going to return
{:ok, "some_token"} | :error

Yes, let's go with this contract and we can mirror it on SQS! :+1: /c @msaraiva

Would it more make sense to deprecate the :scope option, or always pass it as the first argument to the generator MFA?

Good question. It depends if most people will use the Goth one out of the box. If so, I would say the :scope is exclusive to when you pass no token generator. If they pass their own token generator, then they need to pass the scope or whatever else by themselves.

Thanks! I think Goth will be the de facto token provider, but I refused to make it a hard dependency because there are other options out there, and the Google libs don't require it.
I appreciate the input!


Added Chinese Problem

Classic ancient Chinese puzzle solution:
We count 35 heads and 94 legs among the chickens and rabbits in a farm. How many rabbits and how many chickens do we have?

Simple yet unique, nice one!


Changes made in widgets and siddhi files

Purpose

Changes made in IS Analytics widgets and siddhi apps

Please add more information in the PR such as what is the change you have done (Refactoring widgets and add purging logic to session-related tables)


Cannot build without charting_library

Honk I get these errors
Module not found: Can't resolve '../../../public/static/charting_library' in '/vercel/path0/src/frontend/src/components/trade'
--
15:37:43.608 |
15:37:43.608 | Import trace for requested module:
15:37:43.608 | ./src/components/trade/TVChartContainer.tsx
15:37:43.608 |
15:37:43.608 | ./src/pages/market/[market_id].tsx
15:37:43.608 | Module not found: Can't resolve '../../../public/static/charting_library' in '/vercel/path0/src/frontend/src/pages/market'
15:37:43.608 |
15:37:43.609 | ./src/pages/market/[market_id].tsx
15:37:43.609 | Module not found: Can't resolve '../../../public/static/charting_library' in '/vercel/path0/src/frontend/src/pages/market'

@goozeberry is this a build log from Vercel?
Note that if you are trying to build on Vercel, you'll need to set the environment variables: https://github.com/econia-labs/econia-frontend?tab=readme-ov-file#1---environment-preparation
As described on the README, you do not actually need the charting library to build the project, but you'll need to make sure you set TRY_CLONING_TRADINGVIEW to something other than 1

@goozeberry as of #217 you can now fallback on an open source version


Build-qt.sh fixes and updates

Fix installation directory option
Fix and enhance OS X sysroot option
Exit when a command fails
Avoid redownloading libraries when running multiple times, such as during testing
Make output quieter so that errors are more apparent
Disable phonon to support building with OS X 10.12 SDK
Move library versions to configuration variables

@fbudin69500 Could you review this?

@msmolens Looks good to me. Did you run it on travis CI?

Thanks. How do I run it on TravisCI? (Shouldn't that happen automatically?)

I am not sure if there is a hook. If there isn't, you can also add the hook on your personal repo of qt-easy-build, or even run it manually from travisCI

It seems there are some problems with the TravisCI configuration: https://travis-ci.org/msmolens/qt-easy-build/jobs/254929715

Indeed. I had updated this repo a while back so that it would work on travisCI, but clearly it is not up to date anymore :(
Anyway, your patch looks good to me. @jcfr , do you think travis.yml should be updated now?


The frontgate/drone image version should match openpitrix scheduler image version

The frontgate/drone services are started in vmbased runtime's instances, use latest image now.
When openpitrix scheduler modules are upgrading, may add new rpc request from pilot to frontgate, while frontgate uses old image, which has not the new rpc request handler function.
The problem is: how to upgrade frontgate's image to match openpitrix scheduler image version

I found a solution (https://github.com/v2tec/watchtower), and I think we can try it.
It will watching on the container that you cared about, if it's image has changed, it will pull the new image, and restart the container with the same option.

Nice, the solution is good for the situation when use latest image.
But I have another question, if a private openpitrix, use openpitrix v0.2.0, when our frontgate image upgrade to v0.2.4, his/her openpitrix upgrade to v0.2.4, how do his/her frontgate upgrade to v0.2.4.

Pilot, Frontgate, Drone should keep the same version.
When pilot upgrade, frontgate and drone should upgrade too, so frontgate and drone should crontab check if pilot's version is upgraded.
Now need developer put drone into his/her image, drone should also keep the same version with frontgate and pilot. If the sidecar pattern is useful here? Divide into two image, share the file system.

How about this, one container check frontgate or drone has an upgrade binary relaese periodicity, if yes, download it into /opt/openpitrix/bin, and this directory is mounted from /opt/openpitrix/bin, just like below:
docker run -v /opt/openpitrix/bin:/opt/openpitrix/bin updater

Another container mounted the same directory, he will run frontgate or drone from that directory.
docker run -v /opt/openpitrix/bin:/opt/openpitrix/bin metadata /opt/openpitrix/bin/frontgate
# or
docker run -v /opt/openpitrix/bin:/opt/openpitrix/bin metadata /opt/openpitrix/bin/drone

Optimized solution:
1 Pilot serves a rpc request which return its version.
2 Frontgate and drone check the version of pilot through the rpc request periodically. If they found their version not match with the version of pilot. They download new binary release from some url respectively, then they exit.
3 There must be a frontgate or drone exist in their image, maybe located in /usr/local/bin/, and new binary release will be located in /opt/openpitrix/bin. When container is starting, he will check if there is a new release, if so, start the new one, otherwise the origin one.

Optimized solution draft 2:
1 Pilot serves a rpc request which return its version.
2 Frontgate and drone check the version of pilot through the rpc request periodically. If they found their version not match with the version of pilot, then they write the version of pilot to /opt/openpitrix/conf/pilot-version and exit.
3 Frontgate and drone are started by runit, the script used by runit will check if their version is matched with /opt/openpitrix/conf/pilot-version, if not, the script will try to fetch the matched version, and start with the new version.

Merged #624.


Add link to the Demo video on the Readme

Demo (YouTube video)

hey, @ppcano, thanks for the PR
there is a DCO check failing... can you do a git reset --soft HEAD~1 and then commit using git commit -s? thanks

@jainammm, Done

thanks! merging this PR


Questions about Frobenius norm (Table 7)

Hi~
Thanks for your excellent work.
I have a question about the Table 7, where you calculate Frobenius norm. In my view, setting the rank as 4 or 64 only affects $\Delta W_q$, and does not affect $W_q$. The figures at (a) and (b) in the following table are only relevant to $W_q$, since $U, V, W_q$ all come from $W_q$. Therefore, the figures at (a) and (b) should be the same. I could not figure out why they are different.

Is there anything I understood wrong?

Thanks for the question, Zhiyuan.
 That's why they are different even though W_q is the same.

Thanks. It makes perfect sense to me now.

Hi~
Thanks for your help. I read through Sec. 7.3 in detail, and I finally figure out why you use the singular-vector matrix and Frobenius norm. Since I would like to use similar analysis method, I have another little detailed question.
You use $||U^T W_q V^T||$ to measure the scale of $W_q$ projected to the space spanned by $U,V$, where $U,V$ are the left, right singular-vector matrix of $\Delta_W$ or $W_q$.
Here, in SVD decomposition, there are 2 forms of writing: (a) $A = U \sum V$, (b) $A = U \sum V^T$. Since you use $||U^T W_q V^T||$ in the paper ($V$ has a superscript $T$), so I think you should use (a), which means you directly use the right matrix as $V$ without transposition.
Is it right?
Thanks again.

@zhiyuanyou Same question here. Are you figured out?


Make output type stricter and use unknown for error

From my testing the output is always a string array, this just helps confirm that a bit.
I believe unknown is a better typing then error, this will force consumers to do more type checking then just accepting the any.

Thanks for the PR!

From my testing the output is always a string array, this just helps confirm that a bit.

This is true for most language models, but there the output schema depends on the model and can take many different forms.

Hi @johnforte. Thanks for opening this PR!
The output type of a prediction can vary depending on the model. You're correct that many (most?) models output an array of strings. However, some models output an object, array of numbers, or other type. The output type is annotated in the model version's OpenAPI schema. Given that, I think any is a reasonable type.
Alternatively, we could type as either a string, number, boolean, object, array, or null. However, JSON.parse returns any, so that may be an argument in favor of keeping it simple.
As for error, the type of an error is known, but not documented / codified in Replicate's OpenAPI spec. Errors are objects in the shape of an HTTP problem detail.


Fix refresh

I believe this could be a clean fix for the refresh control issue found here: https://github.com/telly/TLYShyNavBar/issues/99
What it does
This checks to see if any of the subviews of the scrollview being set is of type UIRefreshControl.  If it is it sets a new property on the shyscrollviewcontroller equal to that refresh control.  If the refreshControl is nil (a scrollview without pull to refresh) the code behaves the same (still calling tly_setInsets), if the refreshControl exists it only calls tly_setInsets if the refreshControl is hidden. I tried to check if the refreshControl was refreshing but that check is too late.  This PR will make it so when the pull to refresh appears (user is thinking about refreshing) we will no longer layout the views.  It seems to work on my end.

Wow, superb work! I never thought of going with that approach, thanks for submitting this. I'll give it a shot, and merge it ASAP. Hopefully then, make a new release with this being the main new feature :D

Awesome, thanks.  Yea definitely give it a run through, seems to work well for my situation but seeing other situations tested would be great.

Tried this out in my project and it works great.

Good to hear. Thanks for trying it out.


Ensure shape dimensions are within supported integer range (#566)

Proposed changes
In #566, the code below triggered an "unable to allocate" error because the shape's dimension overflowed INT32_MAX.
mx.array(np.arange(1, 30 * 10**8))

Now a more specific error is thrown.
>>> mx.array(np.arange(-1,<PHONE_NUMBER>))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Shape dimension falls outside supported `int` range.

>>> mx.array([0] *<PHONE_NUMBER>)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Shape dimension falls outside supported `int` range.

Note its still possible to produce shapes that trigger an allocation failure.
Checklist

[x] I have read the CONTRIBUTING document
[x] I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
[x] I have added tests that prove my fix is effective or that my feature works
[ ] I have updated the necessary documentation (if needed)

@jmousseau are you coming back to this?

@awni Yes, I should have time to address comments this weekend. Apologies for the slowness/lack of response on my end.


K8S 1.6 another train update from version v1.6.7 to v1.6.8

Fixes #109

This PR needs to be rebased to succeed.


include the osd-network-verifier utility

since manually running the osd-network-verifier is now part of the investigation process for cluster provisioning delays, the network verifier should be part of ocm-container

This is already included in osdctl under osdctl network verify-egress

oh, i see. in that case https://github.com/openshift/osdctl/pull/286


IntelliSense comments don't appear when referencing WinRT winmd version

The DLL versions have IntelliSense comments, but not the WinRT winmd version, despite that version including the XML output.

Someone else reported this issue for their own project: https://social.msdn.microsoft.com/Forums/windowsapps/en-US/638d383e-4185-49f2-87f8-5b15135bb76f/building-a-winrt-component-with-intellisense?forum=winappswithcsharp&prof=required
But the MSDN forums people never replied. And this was all the way back in 2013... Maybe IntelliSense on WinRT simply isn't supported?

It's a known bug with how Visual Studio 2015 grabs the XML comments file.
Workaround is documented here: https://social.msdn.microsoft.com/Forums/en-US/c1168009-b8ae-4bf4-9d4e-85010846b055/building-a-winrt-component-with-intellisense?forum=wpdevelop
I've implemented that workaround in 10240.0.6, which fixed this issue.


Add feature to Create 1 EUR Smart VM - Random DC

Could you please add a feature to allow random data centers which causes the fee to become 1 EUR per month for a small smart vm? Or if there already is a way to do it, please let me know!

Duplicate of https://github.com/Arubacloud/pyArubaCloud/issues/53 :)

Hey @CraftedCat - where have you found this information? I've been checking
https://api.dcx.computing.cloud.it/WsEndUser/v2.9/WsEndUser.svc (with X changed to DC number, like https://api.dc1.computing.cloud.it/WsEndUser/v2.9/WsEndUser.svc) but I haven't found any useful documentation there :)
Have you have this info from the Aruba support?

@Axadiw yes, of course!

Can we hope to add this parameter?)))

I try add:
class SmartVmCreator(Creator):
def init(self, name, admin_password, template_id, auth_obj, note=None):
self.name = name
self.admin_password = admin_password
self.template_id = template_id
self.auth_obj = auth_obj
self.note = 'Create by pyArubaCloud' if note is None else note
self.json_msg = {
'ApplicationId': 'SetEnqueueServerCreation',
'RequestId': 'SetEnqueueServerCreation',
'Username': self.auth_obj.username,
'Password': self.auth_obj.password,
'Server': {
'AdministratorPassword': self.admin_password,
'Name': self.name,
'SmartVMWarePackageID': 1,
'Note': self.note,
'OSTemplateId': self.template_id,
'IsLuckySmart': 1
}
}
but not working...

https://api.dc1.computing.cloud.it/WsEndUser/v2.9/WsEndUser.svc?wsdl IsLuckySmartAvailable

Dear customer
The page shown at this URL is merely intended to provide a few hints on some methods to access the service.
There is no other documentation than the one provide in the Knowledge base at:
https://kb.arubacloud.com/en/api.aspx
We kindly remind we do not provide software assistance, nor support the use of the API.
We had feedback from other customers mentioning the use of tools such as SoapUI (and possibly many similar other tools) to be of great help to visualize the API and available methods (and embedded comments).
Kind regards

Hi,
unfortunately the projects here on github are not up to date considering the improvements we have introduced on the Aruba Cloud platform.
The native api we offer to the end users is based on Soap protocol and is partially described here https://kb.arubacloud.com/en/api.aspx. This set of api is the same we use for public control panels for the our customers.
This is the set available on datacenter IT-1

https://api.dc1.computing.cloud.it/WsEndUser/v2.9/WsEndUser.svc
https://api.dc1.computing.cloud.it/WsCommon/v2.9/WsCommon.svc

Each datacenter provides the same set of api.
In the next months we have in plan to work to the review of our github repository and to provide a better documentation.
Best regards,
Marco

Thanks @marcobarzaghi , we will wait with impatience. But can you tell us fast fix for deploy lucky smart?

@marcobarzaghi I have no right to demand the fulfillment of promises from you. Can you tell what kind of work is underway?))


Unexpected token SyntaxError running tests

First time load of the library on Macbook air.  Just did a fresh install from the latest in the repo; no modifications.  Any ideas what is going wrong here?  Thanks!
Stephens-MacBook-Air:react-testing-components-enzyme slarson$ npm test

><EMAIL_ADDRESS>test /Users/slarson/Documents/react-testing-components-enzyme
> mocha -w test/helpers/browser.js test/*.spec.js

/Users/slarson/Documents/react-testing-components-enzyme/test/helpers/browser.js:9
Object.keys(document.defaultView).forEach((property) => {
                                                     ^^
SyntaxError: Unexpected token =>
    at exports.runInThisContext (vm.js:73:16)
    at Module._compile (module.js:443:25)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at /Users/slarson/Documents/react-testing-components-enzyme/node_modules/mocha/lib/mocha.js:220:27
    at Array.forEach (native)
    at Mocha.loadFiles (/Users/slarson/Documents/react-testing-components-enzyme/node_modules/mocha/lib/mocha.js:217:14)
    at Mocha.run (/Users/slarson/Documents/react-testing-components-enzyme/node_modules/mocha/lib/mocha.js:485:10)
    at loadAndRun (/Users/slarson/Documents/react-testing-components-enzyme/node_modules/mocha/bin/_mocha:359:22)
    at Object.<anonymous> (/Users/slarson/Documents/react-testing-components-enzyme/node_modules/mocha/bin/_mocha:376:3)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3

hey @slarson, I'll take a look. Looks like a transpilation thing at first glance. Actually, no — what version of node are you running?

Thanks for having a look!  I'm running node v0.12.4 with npm v2.10.1 on OSX El Capitan 10.11.6.
Here's what comes out when I do an npm list: http://pastebin.com/spxvwRVE

Fixed when I upgraded node!

dang — didn't respond to this. Yeah, your version of node was pretty old. I'd recommend using nvm if you aren't already 👍


Mounting/Unmounting race condition causing ref to set to null

Given this code:
<MediaQuery minWidth={800}>

 console.log(s)}>800

</MediaQuery>
<MediaQuery maxWidth={799}>

 console.log(s)}>799

</MediaQuery>

The ref will be set to null at last even when div is mounted due to race condition. Using render function doesn't have this problem.
https://codesandbox.io/s/festive-wright-n6pfg?fontsize=14

@kevinlaw91 Not sure what problem do you experience. Given this code:
import React from "react";
import ReactDOM from "react-dom";

import MediaQuery from "react-responsive";

import "./styles.css";

function App() {
  // Try resize the preview panel
  const mobileDiv = React.useRef();
  const desktopDiv = React.useRef();

  const logRefs = () => {
    console.log("mobileDiv", mobileDiv.current);
    console.log("desktopDiv", desktopDiv.current);
  };

  return (

      <MediaQuery minWidth={800}>

desktop

      </MediaQuery>
      <MediaQuery maxWidth={799}>

mobile

      </MediaQuery>
      <button onClick={logRefs}>log refs</button>

  );
}

const rootElement = document.getElementById("root");
ReactDOM.render(<App />, rootElement);

If I resize window then at any point when I click "log refs" button, correct refs are set.

Hi @Tomekmularczyk , I updated the example:
https://codesandbox.io/s/objective-sound-dzu5j?fontsize=14
Basically my use case is the div ref is not 2 different but same component that will share only one ref. The div will be placed differently determined by screen size. In that case the ref will be set to null

@kevinlaw91 the example you provided is throws an error because you try to set ref on function component:

Warning: Function components cannot be given refs. Attempts to access this ref will fail. Did you mean to use React.forwardRef()?
Check the render method of App.
in Foo (created by App)
in MediaQuery (created by MediaQueryContextConsumer)
in MediaQueryContextConsumer (created by App)
in div (created by App)
in App

Oops for some unknown reason the example was not what i saved, sorry for the trouble.
Please check this updated code: https://codesandbox.io/s/charming-resonance-jlebd

I don't think this is a bug in this library. I think its a problem with the usage of refs in this case since when one is removed it will set the ref to null.
To workaround this instead of using 1 ref use two.
const fooRef = React.useRef();
const barRef = React.useRef();

const activeElement = fooRef.current || barRef.current;

I have tried out your solution and it seems to be working fine now. Seems like using multiple refs is the correct way and we should not share one ref with other component. Closing now as this is invalid.
Working example for anyone interested:
https://codesandbox.io/s/loving-ramanujan-ot8x8


Disable search feature

I want to be able to disable search so there is no search frontend.

Pull request #8
Check it out

thanks @wast
OK I'll check it

Merged #8


httpd24 2.4.23

@BrewTestBot test this please

@BrewTestBot test this please

@BrewTestBot test this please

One of these days that bot will give you the answer you're looking for 😉

@kabel yes!! =) We're having some issues with the build bot so that's why I had to rerun it a few times but I'm hoping we can get this merged within the next couple of days.

@BrewTestBot test this please

@kabel thanks for the submission !


FIX : Correct the target platform

Problem
The current target platform selection is wrong.
Solution
Bug fix for target platform priority. Fix the platform list.

The Unity package is different from the .Net package, so I will take that into account.


Change line endings of grammar.js to use LF

Since this was set in .editorconfig, editors would do this automatically. Just thought I'd do this before any other PR to make sure they're not cluttered by this.

Yep, completely makes sense. Merging :smile:
Thanks


A simple starting Point

So I flashed a blinky onto my pico-ice board using SpinalHDL and apio.  Hurrah.  Now I am trying to synthesize the J1SC for the Pico-Ice board.  Oh boy, it is quite tricky.  There is a lot of stuff in there.
What Mecrisp did is to creae a directory skeletalstructure which just had the core parts in it. It was easy to get it up and running, and then one could add things back in one at a time.

It takes a while to read all of the code, figure out what they all do, and figure out how to turn them off.
The problem is that it is lots of changes, and then at the end of the day, when everything is not working, I will not know what I broke.  The ideal is to make it easy to get the first version working on an FPGA.
As a software developer, I would always make lots of changes and then debug it.  As an FPGA developer, they repeatedly instruct us to make one change at a time, get it to work, and then make the next change.
I am not asking that you do anything, I am just recording what might make life easier for those who follow in my footsteps.

On further thought I know what I need.  A little guidance.  The Pico-Ice just has 3 LEDs, red, green and blue, and one button.   Your boards all have a lot more options.  Can I just start with one of your boards, set almost all of the options to null, and everything should work?  Which board is the best one to start with, maybe the most recent software?  Which files should I touch.    Will all of the null options really get ignored, or will it generate an error?
Then once I get the board working, I can create a separate board definition.
I can eventually figure this stuff out, but some guidance, if you have time, would save me much time.

And obviously I have to give it a pcf file.
Here are some minor potential fixes.
If you take a look at your pcf files.
find . -name "*.pcf" ./doc/lattice/icoboard.pcf ./src/main/lattice/IcoBoard/J1Ico.pcf ./src/main/lattice/IceStick/Pins.pcf ./src/main/lattice/IceBreaker/J1Ice.pcf
I presume that you can delete the one in the doc directory.
I would have expected all of the pcf files,  to have the same name, since the directory defines which board they are for.   Or to have all of the pcf files in the same directory, but to give them the board name.
Just minor details for future fixing.  No rush.

So first of all @SteffenReith did say

(have a look at the devel-branch) to learn the features of SpinalHDL and to serve as a starting point and non-trivial example for a Spinal-beginner.

And while i did immediately go and look at it, I  then failed to pop the return stack, read the rest of the sentence, and use it as my starting point.  OOPS!
Next up, if you take a look at that devel branch, the file called
J1Sc/src/main/scala/J1Core.scala  is really very simple.  Somehow much simpler than the original J1 in Verilog.  Not quite sure why the code is so much cleaner, but it is.
The problem is that the scala files J1Ice, J1Config, and J1 add a lot of complexity.   Really all I need are the J1Core, memory and uart interfaces.  Sadly the uart interfaces pull in a whole bunch of other stuff.
I think I need to go into those files, make copies, strip out all but the essential J1 stuff, and then I would have a very simple starting point.
But somehow that is still very scary.    I just want a simple starting point.
One more point.  If we could move the board files to a directory called boards, and the J1 components to a directory called components, or modules, that would be most helpful in reducing the cognitive load.