Unique index / constraint for custom tables are not working

Hey,
I think I've found a bug for the DB migrations part to create a unique field which is not created at all in sqlite (and therefore not working by inserting a new record).
The schema migration files looks as follows:
func init() {
	m.Register(func(db dbx.Builder) error {
		// add up queries...
		// inserts the system profiles collection
		// -----------------------------------------------------------
		profileOwnerRule := fmt.Sprintf("%s = @request.user.id", models.ProfileCollectionUserFieldName)
		collection := &models.Collection{
			Name:       ActivityCollectionName,
			System:     false,
			CreateRule: &profileOwnerRule,
			ListRule:   &profileOwnerRule,
			ViewRule:   &profileOwnerRule,
			UpdateRule: &profileOwnerRule,
			Schema: schema.NewSchema(
				&schema.SchemaField{
					Name:     models.ProfileCollectionUserFieldName,
					Type:     schema.FieldTypeUser,
					Unique:   false,
					Required: true,
					System:   false,
					Options: &schema.UserOptions{
						MaxSelect:     1,
						CascadeDelete: true,
					},
				},
				&schema.SchemaField{
					Name:     ActivityId,
					Type:     schema.FieldTypeNumber,
					Options:  &schema.NumberOptions{},
					Required: true,
					Unique:   true,
				},
				...

My expectation with the param Unique: true is that only 1 x record with that field can be created. Or, on other words, the sqlite index unique is created for this field.
I can add multiple records with the same param for ActivityId. The sqlite index unique is not created.
This shows DBWeaver for the private table _users and the index unique for the field email:

This shows the DBWeaver for the table activities (created in the migration file) and no index unique:

This shows the PocketBase UI with detailed information for the fields (the field activity is labeled as unique) and in the background you see multiple records with the same activityid:

In the repo I see some test for checking the uniqueness for the private tables (like _users) but not for custom tables.
** Update
Sorry, I didn't look enough into the tests. Test as the following clearly defines scenarios for unique:
https://github.com/pocketbase/pocketbase/blob/79265016495c2514a2589b3c79696d9b9e9a8baf/forms/validators/record_data_test.go#L46
However, for me the migration files does not create the index unique.
Hope this helps and I've not done any mistakes. Thanks!
Just for reference, the following is the code to insert the records in an hook event:
rec := models.NewRecord(activityCollection)
		rec.SetDataValue(migrations.ProfileCollectionUserId, userId)
		rec.SetDataValue(migrations.ActivityId, a.Id)
		rec.SetDataValue(migrations.ActivityExternalId, a.ExternalId)
		rec.SetDataValue(migrations.ActivityUploadId, a.UploadId)
		rec.SetDataValue(migrations.ActivityAthleteId, a.Athlete.Id)
		rec.SetDataValue(migrations.ActivityName, a.Name)
		rec.SetDataValue(migrations.ActivityDistance, a.Distance)
		rec.SetDataValue(migrations.ActivityMovingTime, a.MovingTime)
		rec.SetDataValue(migrations.ActivityElapsedTime, a.ElapsedTime)
		rec.SetDataValue(migrations.ActivityTotalElevationGain, a.TotalElevationGain)
		rec.SetDataValue(migrations.ActivityType, a.Type)
		rec.SetDataValue(migrations.ActivityStartDate, a.StartDate)
		rec.SetDataValue(migrations.ActivityStartDateLocal, a.StartDateLocal)
		rec.SetDataValue(migrations.ActivityTimeZone, a.TimeZone)
		rec.SetDataValue(migrations.ActivityStartLatLng, startLatLng)
		rec.SetDataValue(migrations.ActivityEndLatLng, endLatLng)
		rec.SetDataValue(migrations.ActivityMapId, a.Map.Id)
		rec.SetDataValue(migrations.ActivityMapPolyline, a.Map.SummaryPolyline)
		rec.SetDataValue(migrations.Activity, a)
		err = app.Dao().SaveRecord(rec)
		if err != nil {
			fmt.Println(err)
			continue
		}

I'm not sure that I understand what actually the issue is. Constraints and indexes are two different things.
When you mark a field as "Unique" it will add a unique constraint (not index) on application level, aka. you won't be able to submit a new record with the same value. The same apply for the Required constraint.
PocketBase will not change existing user defined data if it is not safe to modify them, aka. if you already have inserted duplicated records, they will remain in the database and the next time you try to edit them and error will be shown that the value already exist.
About the above code - it doesn't do any data validations. If you need validations, the use the forms.RecordUpsert struct.
PocketBase currently doesn't manage add indexes for the created fields - there is already a task to define simple 1 column/field indexes from the Admin UI in the roadmap.

My remark about the constraints vs indexes was to outline the differences between them - while for most dbms adding a unique/foreign key constraint will also add a corresponding index they are usually different things. But that doesn't matter for PocketBase, as you've noted, because we are managing the constraints on application level.
I think I understand now your use case, but the way you are implementing is incorrect.
Executing CREATE UNIQUE INDEX activities_activityid_IDX ON activities (activityid) in a OnBeforeServe hook will run the query every time when your application starts (eg. on process restart).
The Dao is operating directly with the database (aka. doing select, insert, update, delete statements) and it doesn't perform data validations ("exception" to this is when handling the cascade delete operations to ensure that the app is not left in a broken state).
I thinks forms are more suitable for your use case - they can read/load data from the request (or any other external source), validate it and persist (internally they uses the Dao).
So as an alternative, instead of adding a unique sql index, you could replace your Dao().Save call with the form one:
err := forms.NewRecordUpsert(app, rec).Save()

The above should return an error if you have duplicated entries and your field is marked as "Unique".
I completely agree with you that the current code documentation is not enough and we need to create some guides that explain more clearly the internals and their differences. This is exists in the roadmap and will be handled in the near future but I'm currently focusing on stabilizing the PocketBase API since adding more advanced documentation and examples will be hard to keep in sync due to the frequent internal changes.
I'll close the issue for now, but if you are still having difficulties trying to use PocketBase as framework, please let me know and I'll try to help (it also helps me to understand better how users are interpreting the code and whether the current approaches should be reworked).

Hi,
i think it's as follows:
// Works as expected with errors: 'activityid: Value must be unique.'
err := forms.NewRecordUpsert(app, rec).Submit()

// method "Save" does not exist
err := forms.NewRecordUpsert(app, rec).Save()

Thanks!
Maybe a not related question for that. Does a "submit" (create / update) via the forms triggers the realtime subscription? I would like to have a realtime subscription for new records.

Oops, sorry. Yes, it should be Submit(), not Save() (I'll update it in the comments above).
About the realtime subscriptions - currently Create/Update realtime events are send only after the API calls, but I'll make sure to update it in v0.4.0 for consistency with the Delete action (it was recently changed because of #109).
For now, if you want to trigger the realtime subscription events for create/update manually, as a workaround you could trigger the hooks that broadcast the message:
event := &core.RecordCreateEvent{Record: rec}
app.OnRecordAfterCreateRequest().Trigger(event)

event := &core.RecordUpdateEvent{Record: rec}
app.OnRecordAfterUpdateRequest().Trigger(event)

But keep it mind that the above will change in v0.4.0 and the hooks will be replaced with onModelAfterCreate and OnModelAfterUpdate.

Holy! It's working! Thanks for your time and very detailed responses. Awesome work!

@mbecker Just to let you know that with v0.4.0 release changing a record with the Dao will trigger automatically a realtime create/update event so there is no longer need to trigger them manually.

Hi,
Just by curiosity, why did you choose to have contraints on application level?
If we want to use it as a framework, it can cause issues.
For example, If I have a relationship field, let's say device_id, it won't perform the check to see if the device exists if I use the dao.
// I use directly the model to insert a new record
m := &Motions{
  DeviceID: "that doesn't exist"
// ....
}
dao.Save(m)
// it works :(

I need to use forms right? Seems a bit weird to not leverage sql constraints / foreign keys ect.

Dao doesn't perform validations. If you need validations use the forms.
Custom models are just an option (and they need a little more refinement). For most of the cases the Record model should be the preferred way to query and modify your data.

Just by curiosity, why did you choose to have contraints on application level?

Because the constraints are "dynamic" and doesn't require modifying the schema. Unique indexes (and regular indexes in general) will be handled in #1466 and #544.


Another Approach Using normalize

Howdy! I randomly found your package through Bluesky, and wanted to share a different approach we use:
from unicodedata import normalize

def remove_diacritics(text):
    """
    This will remove diacritics such as umlauts and accents from a string. This is
    useful when we need to prepare data for systems such as LDAP.

    Example input: "Carina Müller"
    Example output: "Carina Muller"
    """
    return normalize("NFD", text).encode("ascii", "ignore").decode("utf-8")

This might be more complete than the map you're using? Cheers!

Hey FlipperPA!
Thanks a lot for your contribution! I have just added your suggestion to the library.
Also created the "CONTRIBUTING.md" file to register your name there.
I'm very new on Github and to be honest, I wasn't expecting to have an interaction so soon haha
I have indeed considered a similar approach and I was looking for something that could not change other types of special strings, for example:
remove_diacritics('æ') would return ''
remove_diacritics('北亰') would also return ''
But why not having both options? With this change I have also took the opportunity to improve the library readability.

Oh awesome, that makes sense! It was good to see someone else tackling this issue, when I came up with this solution, there wasn't much out there. I'd rather have UTF-8 support in LDAP and elsewhere, but some places, it just doesn't exist. :) Great work!


Transaction strategy doesn't rollback properly

context
As in the title, it seems that the transaction strategy doesn't rollback correctly. For example
  context 'create user' do
    before do
      users.insert(id: 1)
    end

    it { puts "User count: #{users.count}" }
  end

  context 'create another user' do
    before do
      users.insert(id: 2)
    end

    it { puts "User count: #{users.count}" }
  end

With this configuration
RSpec.configure do |config|
  config.around(:each) do |example|
    conn = DatabaseCleaner[:sequel]
    conn.db = connection
    conn.strategy = :transaction

    conn.start
    example.run
    conn.clean
  end
end

It will produce the result
User count: 1
User count: 2

If I'm not wrong, the expected behavior should be
User count: 1
User count: 1

I tried to mix the deletion and transaction strategy together
RSpec.configure do |config|
  config.around(:each) do |example|
    conn = DatabaseCleaner[:sequel]
    conn.db = connection
    conn.strategy = :transaction
    conn.clean_with(:deletion)

    conn.start
    example.run
    conn.clean
  end
end

And this produces the correct result (User count: 1 for both examples). However, this seems weird that I have to manually use deletion to clean up data. Also, if I was to add a before(:context) setup, then that will wipe out all data I created in the before(:context).
I added a minimal example to reproduce here: https://github.com/Vuta/cleaner_test

Close as it's a problem in my setup. I was using different connections


Fix ARVRServer crash at exit

Fixes  #29272
I don't know why exactly singleton is null, but maybe this code is executed after cleaning some variables and singletons.

CC @BastiaanOlij

Hmm, wonder why indeed, that suggest the drawing loop is still active while
godot is shutting down.
The fix looks fine to me though I wouldn’t mind knowing how it gets cleaned
up prematurely. What platform did you get the crash on?
Cheers,
Bas
On Sat, 6 Jul 2019 at 5:24 am, Rafał Mikrut<EMAIL_ADDRESS>wrote:

CC @BastiaanOlij https://github.com/BastiaanOlij
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/pull/30349?email_source=notifications&email_token=AAO262PW3LCNWJ7ZBUDZLTLP56NVDA5CNFSM4H6NOZ3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZKFT4A#issuecomment-508844528,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAO262M4EOJOMH7KFQM35MLP56NVDANCNFSM4H6NOZ3A
.

--
Kindest regards,
Bastiaan Olij
https://www.facebook.com/bastiaan.olij
https://twitter.com/mux213
https://www.youtube.com/BastiaanOlij
https://www.youtube.com/channel/UCrbLJYzJjDf2p-vJC011lYw
https://github.com/BastiaanOlij

Thanks!


bpf: remap MARK_MAGIC_SNAT_DONE marker to avoid conflicts

Commit f25d8b97e908 ("bpf: Preserve source identity for hairpin via stack")
recently broke BPF NodePort since the use of MARK_MAGIC_SNAT_DONE is now in
conflict/overlapping with MARK_MAGIC_IDENTITY.
In tc egress hooks of bpf_{netdev,overlay} we have a check whether the SNAT
is not needed via if ((ctx->mark & MARK_MAGIC_SNAT_DONE) == MARK_MAGIC_SNAT_DONE).
MARK_MAGIC_SNAT_DONE is 0x0500 whereas MARK_MAGIC_IDENTITY is 0x0F00. So far
it was never a problem since MARK_MAGIC_IDENTITY as not used in a path where
MARK_MAGIC_SNAT_DONE was tested until this changed via f25d8b97e908 where now
SNAT was skipped for Pod traffic.
As a result, we've seen various flakes in SNAT or Hybrid NodePort setups mostly
on the tftp/UDP tests. As a quick fix, remap MARK_MAGIC_SNAT_DONE to 0x1500,
so that setting MARK_MAGIC_IDENTITY will get a mismatch on above mentioned
test and hence we'll end up doing the SNAT. This reaches into MARK_MAGIC_KEY_ID
bit space, but the agent today cannot be configured to run with BPF NodePort
and IPSec at the same time.
Fixes: #10942
Fixes: f25d8b97e908 ("bpf: Preserve source identity for hairpin via stack")
Reported-by: Martynas Pumputis<EMAIL_ADDRESS>Signed-off-by: Daniel Borkmann<EMAIL_ADDRESS>

test-me-please

test-focus K8sService.*

test-me-please

test-focus K8sService.*

Coverage decreased (-0.02%) to 46.776% when pulling f89a91af53f4bf1ecd2ba0d696f004aa7313de3d on pr/fix-snat-marker into 4ce5a4ed9a7ceedf4997a3c2ee895b89bfb58695 on master.

test-gke

(Needs to be backported where also https://github.com/cilium/cilium/pull/10926 gets backported.)


Create the "hiddenCopyElement" in the PDFViewer constructor (PR 16286 follow-up)

To make this functionality work out-of-the-box in custom implementations, see e.g. the "viewer components" examples, it'd be slightly easier if we dynamically create/insert the "hiddenCopyElement" in the PDFViewer constructor.
Given that the "copy all text" feature still appears to work just as before with this patch, hopefully I'm not overlooking any reason why doing this would be a bad idea.

/botio integrationtest


Limits of fields in KongConsumer and KongConsumerGroup

Current Behavior
Limits of Consumers in Konnect:

username's length < 128 and match regex ^[\p{L}\p{N}.\-_~+@/\[\]]+(?: [\p{L}\p{N}.\-_~+@/\[\]]+)*$
custom_id match regex ^[0-9a-zA-Z.\-_~\(\)#%@|+\[\]]+(?: [0-9a-zA-Z.\-_~\\(\)#%@|+\\[\\]]+)*$
Limits of consumer groups in Konnect:
name's length < 128 and match regex ^[\p{N}\p{L}.\-_~]*$

Currently they are not validated on CRD layer or controller layer, and invalid KongConsumer or KongConsumerGroup will be sent to Konnect and get 400 responses. Controller keeps retry in such cases.
Expected Behavior
Either:

Validate and reject invalid instances of KongConsumers and KongConsumerGroups
Or detect the validation errors and stop reconciliations of invalid resources

Steps To Reproduce
Operator Version
kubectl version

... invalid KongConsumer or KongConsumerGroup will be sent to Konnect and get 400 responses. Controller keeps retry in such cases.

With recent introduction of https://github.com/Kong/gateway-operator/blob/73b907ca1bd6eeecd86c2fccc268af35fc5ef4bb/controller/konnect/ops/ops.go#L382 and https://github.com/Kong/gateway-operator/blob/73b907ca1bd6eeecd86c2fccc268af35fc5ef4bb/controller/konnect/ops/ops.go#L177 those should be ignored (i.e. the error shouldn't be reported from the reconcile loop and thus this should not be retried).


Prefilled templates do not pass validation due to character limit

Prefilled templates for sumstats-only submissions now fail validation, due to character limit. We need a solution to allow upload of such templates. In the example attached, the column which exceeds the character limit is "Discovery sample". Note that this data does not exist in the metadata submission template. There is a column called "Sample description" in the metadata template, but this not the same information. The "Discovery sample" field in the database is generated in the curation app either as a combination of sample number (individuals/cases/controls) and ancestry, or entered directly as free text.https://app.zenhub.com/files/2995118/6acf667b-7a47-4aa0-9d02-91501dbeb8fb/download

@ljwh2 @jdhayhurst - I had a look into this and the 'Discovery sample' column in the SUMMARY_STATS template (since this is not a METADATA template corresponds to the sample_description field in the schema that does enforce the 255 field size.
In the template service, this column is only introduced for summary stats submissions. In the Curation App, the import of summary stats submissions does not use this data.
Resolution: I will remove the 255 characters constraint and this should fix things.


Handles request rejection for expired subscriptions

Expired subscriptions were not being checked when data was being sent by / requested from the tower. This PR handled that case and also moves the UserInfo deletion to the Gatekeeper instead of it being handled by either the Watcher or the Responder.
For this changes to work, the Gatekeeper now needs to have a notion of time, which is given by making him subscribe to the ChainMonitor. Also, data is being deleted with a USERS_CACHE_SIZE_BLOCKS delay so both the Watcher and the Responder can query any user related data they may need in order to do their own cleanups.

Reworked the subscription expiry to get rid of the cache as mentioned in https://github.com/talaia-labs/python-teos/pull/231#discussion_r492946714. I've added two separate commits that can be squashed once they're reviewed.

Still missing tests for update_outdated_users_cache, will add them tomorrow morning.
Needs rebasing to add all functionality in a single commit and all tests in another.

Should be good now. Will need rebasing to add all functionality in a single commit and all tests in another once reviewed.


Add test1 by yess

Song Name
test1
Artist Name
yess
Source
Direct file download
Youtube Link
5OGQU-EREqZAXVHg
Direct File Link
https://www.newgrounds.com/audio/listen/1248350.mp3
Song ID
467339
Start Offset [ms]
0
Extra
Created from Auto Nong.
Created for level: 10565740

reject
invalid YouTube link


Scopes adjustments on prod

Env - prod
adjust scopes for HR and ADMIN roles:
UPDATE roles SET scope = 'legal_entity:read employee_request:approve employee_request:reject employee:read division:read division:details declaration_request:write declaration_request:approve declaration_request:reject declaration_request:read declaration:read otp:read otp:write', updated_at = localtimestamp WHERE name = 'ADMIN';
UPDATE roles SET scope = 'legal_entity:read employee_request:read employee_request:write employee_request:approve employee_request:reject employee:read employee:write employee:details employee:deactivate division:write division:read division:details division:activate division:deactivate', updated_at = localtimestamp WHERE name = 'HR';

Done


Centralized logging facility

Currently everything that logs is using Debug.Log. But this is not suitable for core assemblies as this requires a dependency on UnityEngine which isn't appropriate for our preloader support. We need a central logging facility for all core assemblies to use, which pipe back to bepinex.
We should have some kind of abstraction that prefixes Disunity log statements with something like [Disunity][Core] or [Disunity][Runtime] etc.

This was added.


Webhooks no longer available

EventSub replaced the Webhooks system.
The endpoints in the /webhooks namespace are no longer available.
It may be good, to remove them from the lib.

I've added a deprecation warning since removing those methods would require a major version bump. Would be good to combine this with #127


Error: incorrect header check at Zlib._handle.onerror when POST in axios

I setup my own node express application, and when I'm trying to do POST, I got an error when I do a POST from client side
// client side code
axios.post(MY_TEST_URL, data)
    .then(response => {
      console.log(response);
      return response.data
    })

And I got this error from my node express server
{ Error: incorrect header check
    at Zlib._handle.onerror (zlib.js:370:17) errno: -3, code: 'Z_DATA_ERROR', status: 400, statusCode:
 400 } 'Error: incorrect header check\n    at Zlib._handle.onerror (zlib.js:370:17)'

And the following is the header I received in the error handling from my express error handling
{ host: 'localhost:9898',
  connection: 'keep-alive',
  'content-length': '13',
  accept: 'application/json, text/plain, */*',
  origin: 'http://localhost:8080',
  'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36',
  'content-type': 'application/json;charset=UTF-8',
  referer: 'http://localhost:8080/',
  'accept-encoding': 'gzip, deflate',
  'accept-language': 'en-US,en;q=0.8',
  'content-encoding': 'gzip' }

The following is my "Network Tab"
( This one is Preflight Check )

And this is the curl generate from chrome devtool when I observe my frontend application
curl 'http://localhost:9898/api/test' -H 'Content-Encoding: gzip' -H 'Origin: http://localhost:8080' -H
 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36' -H 'Content-Type:
application/json;charset=UTF-8' -H 'Accept: application/json, text/plain, */*' -H 'Referer:
http://localhost:8080/' -H 'Connection: keep-alive' --data<EMAIL_ADDRESS>--compressed

It is totally fine when I do that from POSTMAN.
Any help will be appreciated, thanks.

Compression is done automatically in the browser, so it must be something unrelated to axios. Please test it with a newer version of your browser or maybe check if there's a bug in Node or Express about this.


Will not delete certain ocr'd files

Howdy again,
Seem to be having a strange issue. I take the container down nightly as I do a backup of all of my containers and when i bring it back up in the morning, a couple of the files that had been deleted the previous day are back.  The first time, i just thought it was me, but it has been 4 days now and those 2 files continue to appear daily.   Its weird.  I am not a programmer or anything so let me know what you need to diagnose this.
OS: unRAID 6.8.3 (running docker-compose from the commandline)
PostGRES db container: default from your compose file
worker container version:  eugenci/papermerge-worker:latest
service container version: eugenci/papermerge:1.2.x (and yes it is "x")
Only changes i have made to your compose file are:
changed the line 'depends_on: - db' to 'links: - db:db' (wouldn't talk the other way for some reason)
service version 1.2.x
added exposing ports for postgres (-p 5432:5432 which i had to do to get them to talk to one another for some reason), other than that it is what you wrote.
Also, no errors or anything in the logs. Just files reappearing.
Thanks for looking.

Well written a very clear report!
I will have a look at this issue over the weekend and come back with feedback.

I can't reproduce this issue. IMHO document deletion works pretty well.

Alright. Must just be a strange side effect of the way i am running it.
I'll look into it more and see if i cant figure it out. Thanks!
On Fri, May 8, 2020, 2:28 AM Eugen Ciur<EMAIL_ADDRESS>wrote:

I can't reproduce this issue. IMHO document deletion works pretty well.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ciur/papermerge/issues/14#issuecomment-625657452, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AJPVARLF5R4RCYULEBEB5FDRQORA5ANCNFSM4MZ6TCMQ
.


Building METARs

A useful feature would be to allow the user to build METAR strings from native data types specified through setters.
This would require a rebuild of the METAR string each time a new piece of data is set, or a method of notification when the METAR string is currently out of date and needs rebuilding manually.
The feature would be an extension to the existing implementation of the Metar class which currently only parses a preset string.

Closing as this doesn't fit with current goals.


META6.json is not searchable

But sorry, I'm not sure where to describe the information of the META6.json.
FYI:https://design.perl6.org/S22.html

It's in here http://docs.perl6.org/language/modules#Distributing_Modules but it could be a) more complete and b) searchable

@jonathanstowe
Thanks for the reply !
I have changed the title of the issue.

Closed with https://github.com/perl6/doc/pull/1155.


healthyChecker tcp not work

Please read the CONTRIBUTING.md guidelines to learn on which channels you can
seek for help and ask general questions:
https://github.com/Kong/kong/blob/master/CONTRIBUTING.md#where-to-seek-for-help
Summary
i want to set active healthCheck and checks.active.type=tcp, but it not work.   then  i  found source code in kong/runlooper/balancer.lua
      line 314:
      if (ngx.config.subsystem == "stream" and checks.active.type ~= "tcp")
         or (ngx.config.subsystem == "http" and checks.active.type == "tcp")
      then
        checks = pl_tablex.deepcopy(checks)
        checks.active.healthy.interval = 0
        checks.active.unhealthy.interval = 0
      end

when ngx.config.subsystem=http  then tcp not work,  why do this?
Steps To Reproduce

Additional Details & Logs

Kong version (1.2.0)
Kong debug-level startup logs ($ kong start --vv)
Kong error logs (<KONG_PREFIX>/logs/error.log)
Kong configuration (the output of a GET request to Kong's Admin port - see
https://docs.konghq.com/latest/admin-api/#retrieve-node-information)
Operating system

Hello,
Active Healthchecks over TCP are currently not supported, and this is documented in the docs. We might revisit this in future versions of Kong.


Improve String Comparison Performance & Consistency

Description
Following Microsoft Best Practices documentation there are a number of locations inside of the DNN Platform API's that use ToUpper, ToUpperInvariant, ToLower, ToLowerInvariant and other string comparisons that are against the best practices reccomendations.
These methods should be standardized and the use should be moved to consistently use StringComparison.InvariantCultureIgnoreCase and String.Comparison.OrdinalIgnoreCase (for internal items).
Steps to reproduce
The impacts of this behavior can only be seen with a Memory Profiler or similar tool.
Current result
The existing code results in multiple objects being created adding to the number of items that need to be Garbage Collected when we can simplifiy and optimize the comparisons, resulting in less memory churn which will result in faster performance.
Expected result
When monitoring memory consumption the same string should not be seen as initilaized repeatedly and immediate garbage collected.
Affected version

[x] 9.2.1
[x] 9.2
[x] 9.1.1
[x] 9.1
[x] 9.0

Affected browser
N/A

I believe StringComparison.InvariantCulture and StringComparison.InvariantCultureIgnoreCase are recommended against in the vast majority of use cases.  You pretty much always want Ordinal or OrdinalIgnoreCase if you don't want to use the current culture.
Overall, I'm a big fan of this change 👍

You are correct here, but in the interest of limiting breaking changes if it was a ToLowerInvariant, I keep the Invariant, otherwise doing Ordnial for performance


Undefined variable: hasSudoed

Problem
I just installed this in my Laravel project & I get Undefined variable: hasSudoed the button seems to be down on the right & is working.
Versions
Laravel 5.4
viacreative/sudo-su 1.0

make php artisan vendor:publish
then go to config/sudosu.php , you will find this array
'allowed_tlds' => ['dev', 'local']
append your domain to it ,
like if my url is myapp.dev/user/login i will make this
'allowed_tlds' => ['dev', 'local','myapp.dev']

@Te7a-Houdini Thanks for the reply! I have tried your solution & unfortunately it still presents me with  Undefined variable: hasSudoed.

If I remove
@if (env('APP_DEBUG'))
    @include('sudosu::user-selector')
@endif

the partial from my layout it works & is still there. That's weird the readme says include that in the layout?

@dragonfire1119  try to use the new release  ,and check if that happens again
"viacreative/sudo-su": "^1.1"

Seems to be working for me? It could have something to do with the fact that I just pushed a new release (1.1). Maybe try that version?

@Te7a-Houdini Thanks for the help! @mrterryh Must of been a bug with the 1.0.


Added ability to rescan @Database or @PostgreSQL table list

Validations

[X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that requests the same enhancement

Problem
@Database or @PostgreSQL table list is not updated until IDE is restarted.
Solution
Added feature to refresh @Database or @PostgreSQL table list

As a quick workaround you can use "Developer: Reload Window" via Shift-Cmd-P to basically restart VSCode+Continue, that will also reload/refresh database connections.


⚠ Bofrost.es has degraded performance

In 498a0aa, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1527 ms

Resolved: Bofrost.es performance has improved in 581ecea after 10 minutes.


Allow single-line annotations in parameters/arguments

Resolve #556 and #608.
As I mentioned in the issue in the official styleguide there's nothing about single-line annotations on method/constructor parameters. Also extended it a little, to allow the same thing in arguments' annotations (e.g. when you have a @Retention(SOURCE) @Target(EXPRESSION) for the annotation class)

Yes, I see it is a valid code style. Thank your for contribution :+1:


Set python unicode encoding for check scripts

This PR fix issue with check_ceph_rgw:
$ python --version
Python 2.7.5
$ python check_ceph_rgw
  File "check_ceph_rgw", line 19
SyntaxError: Non-ASCII character '\xc3' in file check_ceph_rgw on line 19, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

I also set encoding for all scripts.

@k0ste thanks for the patch


[URGENT] Crash reported by PlayStore: android.app.RemoteServiceException

android.app.RemoteServiceException
Context.startForegroundService() did not then call Service.startForeground()

Some quick fixes have been implemented, let's close this one and see what happen on version 0.8.25


Document the difference between host and system metrics

This PR aims to add clarifications about the difference between the host and system namespace. This is to address https://github.com/open-telemetry/semantic-conventions/issues/226 and mostly reflects the outcome of the System WG's discussions as described at https://github.com/open-telemetry/semantic-conventions/issues/226#issuecomment-1700529192.

Anything else missing here? Otherwise this can be merged?


ApplicationReferenceRequest is still sent after the reference has been submitted

A second ApplicationReferenceRequest notification is queued to be sent 7 days after the first. If the reference has already been submitted, it should not be sent.
The logic for this has been defined in the 'via' method in /app/Notifications/ApplicationReferenceRequest.php, which is called when queuing the notifications, instead of when the notification is ready to be sent.
This logic should be refactored so that the visit/transfer cron or a queued job (instead of queued notification) makes the check and then sends the reminder notification if the reference hasn't been submitted.

Assiged to @atoff following discussion on #1071.
@atoff It might be easier to add the InteractsWithQueue trait to NotifyAllReferees, and then use $this->release($delay) to delay the job again - when it comes back around, we can then test whether it has been completed or not, and send the notification again.


Add support for course shortcodes

This adds support for course shortcodes. This allows us to have links like edu.cs.illinois.edu/queue/cs225. This link will direct the visitor to the current queue if there is only one queue, or will send them to the course page if the course has multiple queues.
This PR makes the assumption that we'll trust whoever is creating shortcodes (aka admins). To avoid having to duplicate next.js routing behavior and rely on private APIs, the shortcode middleware runs before any next.js route handling. This simplifies things, but could also potentially allow a malicious person to hijack an important route. Thankfully, express only allows [a-zA-Z0-9_] in route params, so this is mostly safe.
On the offchance that we have to revert to the slightly hacky way, the following server.js was working at the time of PR:
/* eslint global-require: "off", no-console: "off" */
const app = require('./app')
const server = require('http').Server(app)
const io = require('socket.io')
const nextJs = require('next')
const co = require('co')

const routes = require('./routes')
const serverSocket = require('./socket/server')
const { baseUrl } = require('./util')

require('dotenv').config()

const DEV = process.env.NODE_ENV !== 'production'
const PORT = process.env.PORT || 3000

const nextApp = nextJs({ dev: DEV })

/* eslint-disable func-names */
co(function* () {
  // Initialize the Next.js app
  yield nextApp.prepare()

  // Websocket stuff
  const socket = io(server, { path: `${baseUrl}/socket.io` })
  serverSocket(socket)

  // A bit ugly, but we need to ensure all other routes are processed before we
  // try to run the URL shortcodes route. We duplicate the
  app.use(async (req, res, next) => {
    const { route, query, parsedUrl } = routes.match(req.url)
    if (route) {
      nextApp.render(req, res, route.page, query)
      return
    }

    if (nextApp.hotReloader) {
      await nextApp.hotReloader.run(req, res)
    }

    const nextFn = nextApp.router.match(req, res, parsedUrl)
    if (nextFn) {
      nextFn()
      return
    }

    next()
  })

  server.listen(PORT)
  console.log(`Listening on ${PORT}`)
}).catch(error => console.error(error.stack))

Fixes #22

@genevievehelsel should probably have a look at it -- I don't have a bigger picture of the whole project, so my review was just "oh, neat logic!"


_IndexAssembler.assemble: optimize using toList(growable: false)

The assemble method is a hotspot for List generation, and a fixed-length list is faster to generate.

[✅] I’ve reviewed the contributor guide and applied the relevant portions to this PR.

@gmpassos @bwilkerson what is the status of this change?

The last update on
https://dart-review.googlesource.com/c/sdk/+/381780
is:
This CL has passed the run

@bwilkerson @scheglov same here, please make a call and either accept (+1 & +2 the CL) or reject the PR. Thanks!


Replace overload constraints with static if

From a recent NG thread by @WalterBright:

Also, as mentioned in the std.algorithm.mutation.remove case, constraints in Phobos often confuse "requirements" with "specializations".
Requirements should be user-facing constraints, while specializations are implementation details better handled with internal static if.

This PR picks a couple of low-hanging fruits in std.algorithm. If this PR got too large, let me know.

Yeah, this is a huge diff. If it's not too much work, could you please split it into multiple commits?

Yes, please split into multiple PRs. It's hard to review a diff this large.

Yeah, this is a huge diff.

Oh sorry. It didn't look to me like this at the time.

If it's not too much work, could you please split it into multiple commits?

Not at all - it's automated


Euclid Algorithm

Description 📜
Explain the brute force to find GCD and the Euclid way to find the GCD with explanation , code and Time Complexity.
##Number Theory : Euclid Algorithm #7383

Type of change 📝

[ ] Au#dio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)
[ ] Vi#deo (Animations, screen-recordings, presentations and regular explanatory films are all possibilties etc)
[x] Doc#umentation (Content Creation in the form of codes or tutorials)
[ ] Other (If you choose other, Please mention changes below)

Domain of Contribution 📊

[ ] Android Dev #(Flutter)
[ ] Android Dev #(Java)
[ ] Android Dev #(Kotlin)
[ ] Backend Dev #(Java)
[ ] Backend Dev #(.NET)
[ ] Backend Dev #(PHP)
[ ] Backend Dev #(Python)
[ ] C/#CPP
[x] Competitive #Programming
[ ] Cyber #Security
[ ] DS#A
[ ] Data#base
[ ] Datascience with #Python
[ ] Datascience with #R
[ ] Frontend Dev #HTML/CSS/JS
[ ] Frontend Dev #React/Angular/Vue
[ ] Go#lang
[ ] Interview #Prep
[ ] Java_#Domain
[ ] JavaScript_#Domain
[ ] ME#RN
[ ] Machine #Learning
[ ] Open #Source
[ ] Python_#Domain
[ ] Ru#st
[ ] Statis#tics
[ ] UI/#UX

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

Screenshots / Gif (Optional) 📸

please review

@ravireddy07 please review

done all the required changes, please review

I have placed the headings and the code is formatted . please review.

@Rishabh510 please review

wait @Rishabh510 , Don't merge!
Seems like you have created another PR with same branch before merging this PR. Hence, you see there two files in this. @iam-sakshi .
Please follow the exact steps to create a PR.

I have deleted it. I am sorry ,I didn't noticed .
But now there is only one file that was approved by @ravireddy07 .
@Rishabh510 please review


Support for Alternative Billing APIs

Discussed in https://github.com/dooboolab-community/react-native-iap/discussions/2678

Originally posted by opentechiz January 24, 2024
Dear,
Can you please help me about this question : Does the lib support the Alternative Billing APIs ? https://developer.android.com/google/play/billing/alternative/alternative-billing-with-user-choice-in-app
Best Regard,
Hoa

We are not supporting this feature. I believe there was an attempt in #2699, but it doesn't seem to be completed. The writer is @Ankit-R.
For more details, you can refer to the following link: Google Play Billing Library Migration.

@hyochan I have updated the user choice billing this PR. #2803  works fine for me.

I think this is an important feature as ios and android will start supporting alternative payments in more regions in the coming future.


Build Error gh-pages

Hello,
I am getting this error, can someone help?
npm WARN<EMAIL_ADDRESS>license should be a valid SPDX license expression
npm ERR! Darwin 16.3.0
npm ERR! argv "/usr/local/Cellar/node/7.3.0/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v7.3.0
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE

npm ERR<EMAIL_ADDRESS>install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script 'node-gyp rebuild'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the contextify package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs contextify
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls contextify
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/niksair/iWork/projects/atlaschain/geojson.io/npm-debug.log
npm ERR! code 1

check debug.log

@tomscholz can you please look into this?

@tomscholz can you please look into this?

Hi Nikhil,
This error is widely reported: have you tried hunting down some of the diagnoses you'll find by googling the error message?

Hi Nikhil,
This error is widely reported: have you tried hunting down some of the diagnoses you'll find by googling the error message?

@tmcw sure. If it is widely reported, why dont you write a page about teaching it down on various OSs? That will be helpful for people.
I will report if anything works for me.
TA.


Inflector future

Hi, I have some thoughts on future changes which I want do discuss (and possibly implement).
1. Inflections ruleset:
As long as we move to normal instantiated object, we may want to create predefined rulesets with different inflection rules. I suggest to change API a bit and introduce Dry::Inflector::Ruleset , suggested usage:
RULESET_ONE = Dry::Inflector::Ruleset.new do
  plural "virus", "viruses"
  singular "thieves", "thief"
end

RULESET_TWO = Dry::Inflector::Ruleset.net do
  load_defaults!
  uncountable "dry-inflector"
end

inflector = Dry::Inflector.new(condition ? RULESET_ONE : RULESET_TWO)

If we just want to use defaults, we can use Dry::Inflector::Ruleset::DEFAULT. The only question is should we include them if nothing passed to constructor?
inflector = Dry::Inflector.new # <= what should we use?

Also, I think we need to keep possibility to just pass a block to constructor and have #inflections method:
inflector = Dry::Inflector.new do
  plural "virus", "viruses"
end

inflector.inflections do
  singular "thieves", "thief"
end

2. Module
In some cases, users may want to just include Dry::Inflector as module (helper) to their own objects. I don't know if this is a good idea, but for the objects with heavy inflector usage or something like views it may be really handy.
If we will decide to implement a module, I also want to discuss possibility of module builder, so we can pick only helpers we need:
class MyClass
  include Dry::Inflector.module(:pluralize, :singularize)
end

Main question here, is how we may configure rulesets? Should we pass ruleset to .module? Or add the #inflection method to instance? What to do when method depends on ruleset and what if not (dasherize for example)?
Note that main interface in this case still Dry::Inflector.new we just add another one API.
So, let's discuss.

Hi @Ptico

Also, I think we need to keep possibility to just pass a block to constructor and have #inflections method

I think this goes against the immutability mindset of dry-rb. Confirm @solnic and @jodosha?
I think it's better to just fire up a new Dry::Inflector instance.
But, this will let us with one problem. If we a have an "already" customized inflector instance that came from outside our code, and we just want to add some more custom rules? This was the problem solved by #inflections with a block.
Perhaps we should have a kind of copy and customize feature to fill in the gap left by inflections method. What do you think?

Inflections ruleset:

I'll have to think more about this. I think this would be the place for internationalization. (although I don't think internationalization is a priority here, as most projects uses english words for tables, code, models and so on).

@Ptico Hi and thanks for starting this discussion.
1. Inflections ruleset:
I think we should put in average developer's shoes. What's the most common operation that they want to do? Probably instantiate the inflector and pluralize a string.
inflector = Dry::Inflector.new
inflector.pluralize("book")

How we can keep this simplicity and make it compatible with advanced configuration cases? IMO, we can change #initialize method like this:
    def initialize(inflections: Inflections.defaults, &blk)
      @inflections = Inflections.build(inflections, &blk)
    end

In this way we can still satisfy the existing use cases:
# 1
Dry::Inflector.new

# 2
Dry::Inflector.new do |inflections|
  inflections.plural "virus", "viruses"
end

But it can second new cases like:
Dry::Inflector.new(inflections: nil) do |inflections|
  inflections.plural "libro", "libri" # Italian words for "book" and "books"
end

Or
italian = Dry::Inflector::Inflections.new do |inflections|
  inflections.plural "libro", "libri"
end

inflector = Dry::Inflector.new(inflections: italian)
inflector.pluralize("libro")

Would this help with I18n?

I vote against an #inflections method that mutates the object after its initialization. 👎  It's again the problem that mutations in the inflection rules may alter the result of a pluralization over the time.
2. Module
I'm not sure I've got the point of this proposal. 😅 Can you please explain what's the purpose of it. Code examples would help a lot. Thanks!

@abinoam ok, let's drop #inflections :)
@jodosha

Can you please explain what's the purpose of it. Code examples would help a lot. Thanks!

This is mostly for helpers and objects with heavy inflector usage:
class PostsView
  include Dry::Inflector.module
end

a(href: "/users/#{underscore(name)}")

From the other side, it requires a lot of effort for such rare use case.

I trying to remember a gem that had this same type of "module builder" strategy. I know I have already used something like this. I think it's cool, useable and easily feasable. 👍

@abinoam It's dry-types 😉
I'm not sure if this is something to support. The receiver class will mixin too many responsibilities: the primary concern (eg the view) and the inflection concern. Using an object (like the actual design)  ease composition over inheritance. WDYT?

I don't think it's a good idea to have Dry::Inflector.module, we should make it clear that dry-inflector provides an object, that you can use as an explicit dependency injected into a constructor. People can use whatever technique they prefer to provide an inflector to their objects. For example in our dry web apps, it'd be one of the "services" that we have, and it'd be automatically provided via auto-injection.

Oh one more thing, we actually have established "Issue Guidelines", and we're not using issues for discussions.

@abinoam It's dry-types 😉

:joy: Yes!!! I was needing some sleep, surely. :joy:

Oh one more thing, we actually have established "Issue Guidelines", and we're not using issues for discussions.

Sorry @solnic, I was not aware of the guideline. No problem at all. Let's move this discussion to discourse. Feel free to enforce the dry-rb community standards. 👍

@abinoam no worries, I forgot to mention that before. I'll add CONTRIBUTING.md to this project too.

@solnic , should I use category "Ideas" for this kind of topic?
Is there a specific tag/category for dry-inflector already?

Moved the inflector ruleset discussion to https://discourse.dry-rb.org/t/dry-inflector-future-api/398?u=abinoam

Move the inflector as module discussion to https://discourse.dry-rb.org/t/dry-inflector-as-module/399

By the way, I'm not being able to ping @Ptico nor @jodosha on discourse forum.

@abinoam I just signed up on the forum. Thanks.


Add tuning doc and update tuning utility

@KKyang Thanks for the changes. But I think I'd still add a link to this guide on the main index.rst page. You can create a new tile called "How to" for it.


assembleRelease Task :notifee_react-native:extractReleaseAnnotations FAILED

Hi everybody i'm trying to build APK using ./gradlew assembleRelease but it fails with this error Task :notifee_react-native:extractReleaseAnnotations FAILED
This is my app config
minSdkVersion = 21 compileSdkVersion = 31 targetSdkVersion = 31
Package.json :
"@notifee/react-native": "^5.7.0",
Error Details :  > Task :notifee_react-native:extractReleaseAnnotations FAILED

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':notifee_react-native:extractReleaseAnnotations'.
Could not resolve all files for configuration ':notifee_react-native:lintClassPath'.
Could not find lint-gradle-30.0.4.jar (com.android.tools.lint:lint-gradle:30.0.4).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/lint/lint-gradle/30.0.4/lint-gradle-30.0.4.jar
Could not find lint-30.0.4.jar (com.android.tools.lint:lint:30.0.4).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/lint/lint/30.0.4/lint-30.0.4.jar
Could not find lint-checks-30.0.4.jar (com.android.tools.lint:lint-checks:30.0.4).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/lint/lint-checks/30.0.4/lint-checks-30.0.4.jar
Could not find lint-api-30.0.4.jar (com.android.tools.lint:lint-api:30.0.4).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/lint/lint-api/30.0.4/lint-api-30.0.4.jar
Could not find intellij-core-30.0.4.jar (com.android.tools.external.com-intellij:intellij-core:30.0.4).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/external/com-intellij/intellij-core/30.0.4/intellij-core-30.0.4.jar
Could not find kotlin-compiler-30.0.4.jar (com.android.tools.external.com-intellij:kotlin-compiler:30.0.4).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/external/com-intellij/kotlin-compiler/30.0.4/kotlin-compiler-30.0.4.jar
Could not find uast-30.0.4.jar (com.android.tools.external.org-jetbrains:uast:30.0.4).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/external/org-jetbrains/uast/30.0.4/uast-30.0.4.jar
Could not find aapt2-proto-4.1.0-alpha01-6193524.jar (com.android.tools.build:aapt2-proto:4.1.0-alpha01-6193524).
Searched in the following locations:
https://dl.google.com/dl/android/maven2/com/android/tools/build/aapt2-proto/4.1.0-alpha01-6193524/aapt2-proto-4.1.0-alpha01-6193524.jar

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
*Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 7s

Stuck here please help me out

minSdkVersion = 21 compileSdkVersion = 31 targetSdkVersion = 31

You must have compileSdkVersion 33 at minimum.
That implies JDK 11 and 33 build tools as well.
You can leave minSdkVersion alone
You can targetSdkVersion alone if you are not prepared to change it yet (thus not handling notification permission in Android 13)
This is clearly mentioned in the v5 release notes but is not documented in the installation steps, so that may have caught you.

Finally fixed by upgrading Gradle
upgrade classpath("com.android.tools.build:gradle:7.0.4") to classpath("com.android.tools.build:gradle:7.3.0") along with compileSdkVersion = 33 , targetSdkVersion = 33

Yes i tried 7.1 & 7.2. there is no problem with this versions.
maybe we should consider 7.1 as the minimum .

Thank you! Okay, so gradle plugin 7.1 minimum, compileTargetSdkVersion 33 minimum / JDK11 minimum / so gradle minimum is 7.2 (https://developer.android.com/studio/releases/gradle-plugin#updating-gradle)
Now we know the matrix that will work

I have made this change here https://github.com/invertase/notifee/commit/e61114c2281fe593e0de8e097f7a2b7f85c83c8b - @helenaford not sure if the docs will publish anymore there, following #559 ?

@mikehardy no, sadly they won't be until we migrate the docs website over to vercel.


No output in terminal

I'm currently trying to run the application and found myself unable to get anything displayed on the terminal.
My system is win-x64

you can change  color


[BUG] - Inserting New Mappings into node_groups leads to ddestructive behavior

Describe the bug
Modify a Nebari AWS deployment node group definition like so:
# old
amazon_web_services:
  node_groups:
    grp_1:
      ...
    grp_2:
      ...
    grp_3:
      ...

# new
amazon_web_services:
  node_groups:
    grp_1:
      ...
    grp_2:
      ...
    grp_4:
      ...
    grp_5:
      ...
    grp_3:
      ...

Nebari destroyed and recreated the third item in the mapping (so grp_3 became grp_4) then created a new fourth and fifth item
Expected behavior
Create new grp_4 and grp_5 while leaving grp_3 unchanged
OS and architecture in which you are running Nebari
MacOS / ARM Apple Silicon
How to Reproduce the problem?
Make a change in a nebari-config such as:
# old
amazon_web_services:
  node_groups:
    grp_1:
      ...
    grp_2:
      ...
    grp_3:
      ...

# new
amazon_web_services:
  node_groups:
    grp_1:
      ...
    grp_2:
      ...
    grp_4:
      ...
    grp_5:
      ...
    grp_3:
      ...

Command output
Two new node groups were added in the middle of the mapping in the example below.  But see "forces replacement" notes moving the instance types around and so destroying/recreating groups

[terraform]:
[terraform]:   # module.kubernetes.aws_eks_node_group.main[3] must be replaced
[terraform]: -/+ resource "aws_eks_node_group" "main" {
[terraform]:       ~ ami_type               = "AL2_x86_64_GPU" -> (known after apply)
[terraform]:       ~ arn                    = "arn:aws-us-gov:eks:us-gov-west-1:xxxxxxxxxxxxxx:nodegroup/nebari-xxxxxxxxxxxxxx/gpu-tesla-g4/6ec71764-4312-3023-2f8f-da6f2d90d0eb" -> (known after apply)
[terraform]:       ~ capacity_type          = "ON_DEMAND" -> (known after apply)
[terraform]:       ~ id                     = "nebari-xxxxxxxxxxxxxx:gpu-tesla-g4" -> (known after apply)
[terraform]:       ~ instance_types         = [ # forces replacement
[terraform]:           - "g4dn.xlarge",
[terraform]:           + "r5.xlarge",
[terraform]:         ]
[terraform]:       ~ labels                 = {
[terraform]:           ~ "dedicated" = "gpu-tesla-g4" -> "r5-xlarge"
[terraform]:         }
[terraform]:       ~ node_group_name        = "gpu-tesla-g4" -> "r5-xlarge" # forces replacement
[terraform]:       + node_group_name_prefix = (known after apply)
[terraform]:       ~ release_version        = "1.29.3-20240506" -> (known after apply)
[terraform]:       ~ resources              = [
[terraform]:           - {
[terraform]:               - autoscaling_groups              = [
[terraform]:                   - {
[terraform]:                       - name = "eks-gpu-tesla-g4-6ec71764-4312-3023-2f8f-da6f2d90d0eb"
[terraform]:                     },
[terraform]:                 ]
[terraform]:               - remote_access_security_group_id = ""
[terraform]:             },
[terraform]:         ] -> (known after apply)
[terraform]:       ~ status                 = "ACTIVE" -> (known after apply)
[terraform]:       ~ tags                   = {
[terraform]:             "Environment"                                             = "xxxxxx"
[terraform]:             "Owner"                                                   = "terraform"
[terraform]:             "Project"                                                 = "nebari-xxxxxx"
[terraform]:           ~ "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g4" -> "r5-xlarge"
[terraform]:             "propagate_at_launch"                                     = "true"
[terraform]:         }
[terraform]:       ~ tags_all               = {
[terraform]:           ~ "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g4" -> "r5-xlarge"
[terraform]:             # (4 unchanged elements hidden)
[terraform]:         }
[terraform]:       ~ version                = "1.29" -> (known after apply)
[terraform]:         # (4 unchanged attributes hidden)
[terraform]:
[terraform]:       - update_config {
[terraform]:           - max_unavailable            = 1 -> null
[terraform]:           - max_unavailable_percentage = 0 -> null
[terraform]:         }2
[terraform]:
[terraform]:         # (1 unchanged block hidden)
[terraform]:     }
[terraform]:
[terraform]:   # module.kubernetes.aws_eks_node_group.main[4] must be replaced
[terraform]: -/+ resource "aws_eks_node_group" "main" {
[terraform]:       ~ ami_type               = "AL2_x86_64_GPU" -> (known after apply)
[terraform]:       ~ arn                    = "arn:aws-us-gov:eks:us-gov-west-1:xxxxxxxxxxxxxx:nodegroup/nebari-xxxxxxxxxxxxxx/gpu-tesla-g4-4x/bec71764-4314-8153-2845-0b9137a886ea" -> (known after apply)
[terraform]:       ~ capacity_type          = "ON_DEMAND" -> (known after apply)
[terraform]:       ~ id                     = "nebari-xxxxxxxxxxxxxx:gpu-tesla-g4-4x" -> (known after apply)
[terraform]:       ~ instance_types         = [ # forces replacement
[terraform]:           - "g4dn.12xlarge",
[terraform]:           + "r5.2xlarge",
[terraform]:         ]
[terraform]:       ~ labels                 = {
[terraform]:           ~ "dedicated" = "gpu-tesla-g4-4x" -> "r5-2xlarge"
[terraform]:         }
[terraform]:       ~ node_group_name        = "gpu-tesla-g4-4x" -> "r5-2xlarge" # forces replacement
[terraform]:       + node_group_name_prefix = (known after apply)
[terraform]:       ~ release_version        = "1.29.3-20240506" -> (known after apply)
[terraform]:       ~ resources              = [
[terraform]:           - {
[terraform]:               - autoscaling_groups              = [
[terraform]:                   - {
[terraform]:                       - name = "eks-gpu-tesla-g4-4x-bec71764-4314-8153-2845-0b9137a886ea"
[terraform]:                     },
[terraform]:                 ]
[terraform]:               - remote_access_security_group_id = ""
[terraform]:             },
[terraform]:         ] -> (known after apply)
[terraform]:       ~ status                 = "ACTIVE" -> (known after apply)
[terraform]:       ~ tags                   = {
[terraform]:             "Environment"                                             = "xxxxxx"
[terraform]:             "Owner"                                                   = "terraform"
[terraform]:             "Project"                                                 = "nebari-xxxxxx"
[terraform]:           ~ "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g4-4x" -> "r5-2xlarge"
[terraform]:             "propagate_at_launch"                                     = "true"
[terraform]:         }
[terraform]:       ~ tags_all               = {
[terraform]:           ~ "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g4-4x" -> "r5-2xlarge"
[terraform]:             # (4 unchanged elements hidden)
[terraform]:         }
[terraform]:       ~ version                = "1.29" -> (known after apply)
[terraform]:         # (4 unchanged attributes hidden)
[terraform]:
[terraform]:       - update_config {
[terraform]:           - max_unavailable            = 1 -> null
[terraform]:           - max_unavailable_percentage = 0 -> null
[terraform]:         }
[terraform]:
[terraform]:         # (1 unchanged block hidden)
[terraform]:     }
[terraform]:
[terraform]:   # module.kubernetes.aws_eks_node_group.main[5] must be replaced
[terraform]: -/+ resource "aws_eks_node_group" "main" {
[terraform]:       ~ ami_type               = "AL2_x86_64_GPU" -> (known after apply)
[terraform]:       ~ arn                    = "arn:aws-us-gov:eks:us-gov-west-1:xxxxxxxxxxxxxx:nodegroup/nebari-xxxxxxxxxxxxxx/gpu-tesla-g3-2x/eec71764-4316-dcb9-a3ee-94b286f4afff" -> (known after apply)
[terraform]:       ~ capacity_type          = "ON_DEMAND" -> (known after apply)
[terraform]:       ~ id                     = "nebari-xxxxxxxxxxxxxx:gpu-tesla-g3-2x" -> (known after apply)
[terraform]:       ~ instance_types         = [ # forces replacement
[terraform]:           - "g3.8xlarge",
[terraform]:           + "g4dn.xlarge",
[terraform]:         ]
[terraform]:       ~ labels                 = {
[terraform]:           ~ "dedicated" = "gpu-tesla-g3-2x" -> "gpu-tesla-g4"
[terraform]:         }
[terraform]:       ~ node_group_name        = "gpu-tesla-g3-2x" -> "gpu-tesla-g4" # forces replacement
[terraform]:       + node_group_name_prefix = (known after apply)
[terraform]:       ~ release_version        = "1.29.3-20240506" -> (known after apply)
[terraform]:       ~ resources              = [
[terraform]:           - {
[terraform]:               - autoscaling_groups              = [
[terraform]:                   - {
[terraform]:                       - name = "eks-gpu-tesla-g3-2x-eec71764-4316-dcb9-a3ee-94b286f4afff"
[terraform]:                     },
[terraform]:                 ]
[terraform]:               - remote_access_security_group_id = ""
[terraform]:             },
[terraform]:         ] -> (known after apply)
[terraform]:       ~ status                 = "ACTIVE" -> (known after apply)
[terraform]:       ~ tags                   = {
[terraform]:             "Environment"                                             = "xxxxxx"
[terraform]:             "Owner"                                                   = "terraform"
[terraform]:             "Project"                                                 = "nebari-xxxxxx"
[terraform]:           ~ "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g3-2x" -> "gpu-tesla-g4"
[terraform]:             "propagate_at_launch"                                     = "true"
[terraform]:         }
[terraform]:       ~ tags_all               = {
[terraform]:           ~ "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g3-2x" -> "gpu-tesla-g4"
[terraform]:             # (4 unchanged elements hidden)
[terraform]:         }
[terraform]:       ~ version                = "1.29" -> (known after apply)
[terraform]:         # (4 unchanged attributes hidden)
[terraform]:
[terraform]:       - update_config {
[terraform]:           - max_unavailable            = 1 -> null
[terraform]:           - max_unavailable_percentage = 0 -> null
[terraform]:         }
[terraform]:
[terraform]:         # (1 unchanged block hidden)
[terraform]:     }
[terraform]:
[terraform]:   # module.kubernetes.aws_eks_node_group.main[6] will be created
[terraform]:   + resource "aws_eks_node_group" "main" {
[terraform]:       + ami_type               = (known after apply)
[terraform]:       + arn                    = (known after apply)
[terraform]:       + capacity_type          = (known after apply)
[terraform]:       + cluster_name           = "nebari-xxxxxxxxxxxxxx"
[terraform]:       + disk_size              = 50
[terraform]:       + id                     = (known after apply)
[terraform]:       + instance_types         = [
[terraform]:           + "g4dn.12xlarge",
[terraform]:         ]
[terraform]:       + labels                 = {
[terraform]:           + "dedicated" = "gpu-tesla-g4-4x"
[terraform]:         }
[terraform]:       + node_group_name        = "gpu-tesla-g4-4x"
[terraform]:       + node_group_name_prefix = (known after apply)
[terraform]:       + node_role_arn          = "arn:aws-us-gov:iam::xxxxxxxxxxxxxx:role/nebari-xxxxxxxxxxxxxx-eks-node-group-role"
[terraform]:       + release_version        = (known after apply)
[terraform]:       + resources              = (known after apply)
[terraform]:       + status                 = (known after apply)
[terraform]:       + subnet_ids             = [
[terraform]:           + "subnet-xxxxxxxxxxxxxxxxxx",
[terraform]:           + "subnet-xxxxxxxxxxxxxxxxxx",
[terraform]:         ]
[terraform]:       + tags                   = {
[terraform]:           + "Environment"                                             = "xxxxxx"
[terraform]:           + "Owner"                                                   = "terraform"
[terraform]:           + "Project"                                                 = "nebari-xxxxxx"
[terraform]:           + "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g4-4x"
[terraform]:           + "propagate_at_launch"                                     = "true"
[terraform]:         }
[terraform]:       + tags_all               = {
[terraform]:           + "Environment"                                             = "xxxxxx"
[terraform]:           + "Owner"                                                   = "terraform"
[terraform]:           + "Project"                                                 = "nebari-xxxxxx"
[terraform]:           + "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g4-4x"
[terraform]:           + "propagate_at_launch"                                     = "true"
[terraform]:         }
[terraform]:       + version                = (known after apply)
[terraform]:
[terraform]:       + scaling_config {
[terraform]:           + desired_size = 0
[terraform]:           + max_size     = 5
[terraform]:           + min_size     = 0
[terraform]:         }
[terraform]:     }
[terraform]:
[terraform]:   # module.kubernetes.aws_eks_node_group.main[7] will be created
[terraform]:   + resource "aws_eks_node_group" "main" {
[terraform]:       + ami_type               = (known after apply)
[terraform]:       + arn                    = (known after apply)
[terraform]:       + capacity_type          = (known after apply)
[terraform]:       + cluster_name           = "nebari-xxxxxxxxxxxxxx"
[terraform]:       + disk_size              = 50
[terraform]:       + id                     = (known after apply)
[terraform]:       + instance_types         = [
[terraform]:           + "g3.8xlarge",
[terraform]:         ]
[terraform]:       + labels                 = {
[terraform]:           + "dedicated" = "gpu-tesla-g3-2x"
[terraform]:         }
[terraform]:       + node_group_name        = "gpu-tesla-g3-2x"
[terraform]:       + node_group_name_prefix = (known after apply)
[terraform]:       + node_role_arn          = "arn:aws-us-gov:iam::xxxxxxxxxxxxxx:role/nebari-xxxxxxxxxxxxxx-eks-node-group-role"
[terraform]:       + release_version        = (known after apply)
[terraform]:       + resources              = (known after apply)
[terraform]:       + status                 = (known after apply)
[terraform]:       + subnet_ids             = [
[terraform]:           + "subnet-xxxxxxxxxxxxxxxxxx",
[terraform]:           + "subnet-xxxxxxxxxxxxxxxxxx",
[terraform]:         ]
[terraform]:       + tags                   = {
[terraform]:           + "Environment"                                             = "xxxxxx"
[terraform]:           + "Owner"                                                   = "terraform"
[terraform]:           + "Project"                                                 = "nebari-xxxxxx"
[terraform]:           + "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g3-2x"
[terraform]:           + "propagate_at_launch"                                     = "true"
[terraform]:         }
[terraform]:       + tags_all               = {
[terraform]:           + "Environment"                                             = "xxxxxx"
[terraform]:           + "Owner"                                                   = "terraform"
[terraform]:           + "Project"                                                 = "nebari-xxxxxx"
[terraform]:           + "k8s.io/cluster-autoscaler/node-template/label/dedicated" = "gpu-tesla-g3-2x"
[terraform]:           + "propagate_at_launch"                                     = "true"
[terraform]:         }
[terraform]:       + version                = (known after apply)
[terraform]:
[terraform]:       + scaling_config {
[terraform]:           + desired_size = 0
[terraform]:           + max_size     = 5
[terraform]:           + min_size     = 0
[terraform]:         }
[terraform]:     }
[terraform]:
[terraform]: Plan: 6 to add, 0 to change, 3 to destroy.

### Versions and dependencies used.

Nebari 2024.9.1

### Compute environment

None

### Integrations

_No response_

### Anything else?

Digging into the code, it looks like var.node_groups is a TF list type and being created with a count block.  TF's documentation about using for_each instead of count [references these "churn" problems as a reason ](https://developer.hashicorp.com/terraform/language/meta-arguments/count#when-to-use-for_each-instead-of-count)to use for_each

One major headache to taking this approach is that it would difficult to avoid it becoming destructive... a moved block would have to take into account the old list indicies vs new naming convention and I find it highly unlikely it would be generated on the fly.  Perhaps an upgrade script to manually move the state around that includes some logical checking to made sure old->new is as expected would work

The other way to fix this would be through documentation

Thanks for opening the issue, @kenafoster. I think the name of the issue is a bit misleading, though. What leads to the destruction and recreation of resources is not the inclusion of new nodes to the mapping but the rearranging of the order.
This is an interesting finding. Indeed, it is due to us using the count object to dynamically create new node_groups under the Terraform HCL code.
https://github.com/nebari-dev/nebari/blob/e26e6051f587323937769f6cf0fd105bc3591189/src/_nebari/stages/infrastructure/template/aws/modules/kubernetes/main.tf#L71-L74
Hence, each new node_group created by terraform is then referenced by its resources name e.g. aws_eks_node_group.main[0], which, using the example you provide,d would generate the following correspondence:
    grp_1 --> aws_eks_node_group.main[0]
    grp_2 --> aws_eks_node_group.main[1]
    grp_3 --> aws_eks_node_group.main[2]

However, as mentioned above, we would have a new arrangement when there is a change. The main problem is that the already created resource aws_eks_node_group.main[n] will have its metadata changed (e.g., name, limits, etc.), which leads to re-creation.
As you might have considered, the best approach would be to use the for_each block instead since we can loop over the node_group names as the indexer instead of an arbitrary position.


Container name conflicts with docker compose

Describe the bug
When running a compose project (currently using docker compose watch but I have seen it infrequently with normal up) I'm intermittently seeing docker complain about container name conflicts when rebuilding a container. Ctrl+C and running it again will get it to work.
All the containers have a name in the compose file (container_name: auth in this case) but there is 12 hex chars being prepended to it in the error message.
There are about 15 containers defined in total.
service "customer-auth" successfully built
Failed to recreate service after update. Error: Error response from daemon: Conflict. The container name "/f0e73909e438_auth" is already in use by container "802e7e578db73b70fa49d1be51f58a5c200080ccd63da064f65641ca68b44a95". You have to remove (or rename) that container to be able to reuse that name.
WARN[0194] Error handling changed files for service customer-auth: Error response from daemon: Conflict. The container name "/f0e73909e438_auth" is already in use by container "802e7e578db73b70fa49d1be51f58a5c200080ccd63da064f65641ca68b44a95". You have to remove (or rename) that container to be able to reuse that name.
^Ccanceled

To Reproduce
I don't have an isolated test case.
Expected behavior
It should build the container every time, currently it can randomly fail to build (and work on retry).
Diagnostic report (REQUIRED)
OrbStack info:
Version: 1.7.4
Commit: e03df71b90121e11c379c6273bd0a63166b33957 (v1.7.4)
System info:
macOS: 14.5 (23F79)
CPU: arm64, 10 cores
CPU model: Apple M1 Max
Model: MacBookPro18,4
Memory: 32 GiB
Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2024-09-30T21-28-28.514789Z.zip
Screenshots and additional context (optional)
There are multiple compose files being merged for running, but here is this service from the docker compose config output
  auth:
    build:
      context: /Users/elliot/repos/core
      dockerfile: containers/auth/Dockerfile
      x-bake:
        tags:
          - local/auth:GIT-
    develop:
      watch:
        - path: /Users/elliot/repos/core/containers/auth
          action: rebuild
    container_name: auth
    environment:
      AUTH_MODE: backoffice
    expose:
      - "4181"
    healthcheck:
      test:
        - CMD-SHELL
        - curl -f http://localhost:4181/healthz
      timeout: 3s
      interval: 5s
      start_period: 5s
    image: local/auth:latest
    networks:
      default: null
    volumes:
      - type: bind
        source: /Users/elliot/repos/core/cloud
        target: /opt/COMPANY/core/cloud
        bind:
          create_host_path: true
      - type: bind
        source: /Users/elliot/repos/vault
        target: /opt/COMPANY/vault
        bind:
          create_host_path: true```

This looks to be an upstream bug, see https://github.com/docker/compose/issues/11151.


Add support for sending in-progress messages to main thread

Currently, when using Aioli to send a message to a WebWorker, the WebWorker can only return data once before the Promise is resolved. However, there are many cases in which it would be useful for the WebWorker to regularly send status updates (e.g. send updates during a long computation) and only resolve the Promise once the last message is sent.
Implementation Sketch: Support sending 3 types of messages from WebWorker to main thread. Currently accept: resolve or reject; should also accept callback, which doesn't resolve the Promise.

In progress: branch issue-18-support-in-progress-messages


Clicking on a result from homepage leads to a partially loaded page

Whenever I click on one of the (entrance?) results from the homepage, I am lead to a partially loaded page in about same location. The top few results on my homepage I tried are listed from my "Current Location" at 225 N. Michigan.
Here is when I click on Irene C. Hernandez Family Picnic Area from the homepage:

LaBagh Woods:

Forest Glen Woods:

Whenever I exit out of that location result and click another result I go to a correct, properly loaded page. See here:

This is really weird. What browser? Windows or mac? @smarziano

I have done a few tests. This same error happens both on Windows and Mac on these browsers: Chrome, Internet Explorer, and Firefox.

Centering is not working right at first but then does

Fixed with commit https://github.com/smartchicago/trailsy/commit/7921ad7f296b6fd3205ccc1060cc69e755ed4e93

@shua123 This is still an issue for locations that are further away. Here is an example of clicking on "Tower Road" with current location set at 225 N. Michigan Ave:

Here is another example where the map will not load but the trails do for "Cap Sauers Holdings Nature Preserve:

The issue appears with the first movement after load. Part of the transform pan does not reset correctly. There seems to be a variety of fixes integrated into Leaflet over time but nothing that solves the issue completely. https://www.google.com/?gws_rd=ssl#q=leaflet+tilelayer+map+freeze
Disabling 3D appears to "solve" the issue by avoiding it. The downside is that we lose the pan/zoom animation. Ideally we find a better solution later.
@smarziano Can you please check if it works better now for you?

@shua123 This is working much better now. I've tried multiple results (especially some of the further distance results) and all of them are loading properly. I did not notice a difference or that we lost the pan/zoom animation.

:trophy:


Allow Symbol as action name

Hi!
When I'm trying to use symbol as action name I receive a this error:
TS2678: Type 'symbol' is not comparable to type 'string'.
due to this interface definition:
https://github.com/ngrx/store/blob/master/src/dispatcher.ts#L4
export interface Action {
  type: string;
  payload?: any;
}

I think it's reasonable to allow action names to be Symbols, not only strings.
Thank you.

Hey, is anyone interested in this feature?)

hi @govorov - we're migrating over to https://github.com/ngrx/platform. Feel free to re-open this over there if it's still a feature you need. Thanks!


[MATLAB] Add arrow.array.BooleanArray class

Describe the enhancement requested
To continue building out the concrete arrow.array.Array subclasses, now that the MATLAB interface supports validity bitmaps and bit packing/unpacking (#35598), it would make sense to add a arrow.array.BooleanArray class.
BooleanArray would map to the MATLAB logical type when calling toMATLAB.
Component(s)
MATLAB

take


Kubernetes rc, pods are lost after turning off and turning on the host

Rancher Version:: Master
Environment Type: (Cattle/Kubernetes/Swarm/Mesos): Kubernetes
Steps to Reproduce:

Create a Kubernetes environment with 4 hosts. Use library qa-catalog: https://github.com/rancher/qa-catalog/
Create pods, rc
Turn off the host on which the pods, etcd exist. The etcd gets created on another host
Turn on the host which was switched off in step 3.

Results:
The pods, rc, service are all lost.  kubectl get rc does not return anything
> kubectl get rc
>

kubectl get nodes returns only one host instead of 4
> kubectl get nodes
NAME                       STATUS    AGE
soumyamasterkubeenvhost1   Ready     36m

The following errors are seen in the controller manager:
6/27/2016 2:18:42 PMI0627 21:18:42.193864       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:18:42 PMI0627 21:18:42.193973       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:18:47 PMI0627 21:18:47.309505       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:18:47 PMI0627 21:18:47.309603       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:18:52 PMI0627 21:18:52.383838       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:18:52 PMI0627 21:18:52.383865       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:18:57 PMI0627 21:18:57.459911       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:18:57 PMI0627 21:18:57.459941       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:19:02 PMI0627 21:19:02.538104       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:19:02 PMI0627 21:19:02.538136       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:19:07 PMI0627 21:19:07.620104       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:19:07 PMI0627 21:19:07.620140       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:19:12 PMI0627 21:19:12.697697       1 nodecontroller.go:478] Evicting pods on node soumyamasterkubeenvhost1: 2016-06-27 21:19:12.697662593 +0000 UTC is later than 2016-06-27 21:14:48.007726723 +0000 UTC + 4m20s
6/27/2016 2:19:12 PMI0627 21:19:12.697904       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:19:12 PMI0627 21:19:12.697923       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:19:12 PMI0627 21:19:12.778646       1 event.go:211] Event(api.ObjectReference{Kind:"Node", Namespace:"", Name:"soumyamasterkubeenvhost1", UID:"soumyamasterkubeenvhost1", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeletingAllPods' Node soumyamasterkubeenvhost1 event: Deleting all Pods from Node soumyamasterkubeenvhost1.
6/27/2016 2:19:12 PMI0627 21:19:12.778748       1 event.go:211] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"k8testrc-eaazc", UID:"9ae67bd0-3cab-11e6-a0d5-02e309181521", APIVersion:"v1", ResourceVersion:"289", FieldPath:""}): type: 'Normal' reason: 'NodeControllerEviction' Marking for deletion Pod k8testrc-eaazc from Node soumyamasterkubeenvhost1
6/27/2016 2:19:12 PMI0627 21:19:12.786550       1 event.go:211] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"k8testrc-qvdsf", UID:"9ae67cfe-3cab-11e6-a0d5-02e309181521", APIVersion:"v1", ResourceVersion:"290", FieldPath:""}): type: 'Normal' reason: 'NodeControllerEviction' Marking for deletion Pod k8testrc-qvdsf from Node soumyamasterkubeenvhost1
6/27/2016 2:19:12 PMI0627 21:19:12.796410       1 event.go:211] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"k8testrc1-1o6oc", UID:"9843a14f-3caa-11e6-a0d5-02e309181521", APIVersion:"v1", ResourceVersion:"292", FieldPath:""}): type: 'Normal' reason: 'NodeControllerEviction' Marking for deletion Pod k8testrc1-1o6oc from Node soumyamasterkubeenvhost1
6/27/2016 2:19:12 PMI0627 21:19:12.808196       1 event.go:211] Event(api.ObjectReference{Kind:"ReplicationController", Namespace:"default", Name:"k8testrc", UID:"9ae52d58-3cab-11e6-a0d5-02e309181521", APIVersion:"v1", ResourceVersion:"230", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: k8testrc-k5b6j
6/27/2016 2:19:12 PME0627 21:19:12.863320       1 nodecontroller.go:251] unable to evict node "soumyamasterkubeenvhost1": pods "k8testrc1-pnxp2" not found
6/27/2016 2:19:12 PMI0627 21:19:12.866573       1 event.go:211] Event(api.ObjectReference{Kind:"ReplicationController", Namespace:"default", Name:"k8testrc", UID:"9ae52d58-3cab-11e6-a0d5-02e309181521", APIVersion:"v1", ResourceVersion:"230", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: k8testrc-ffvby
6/27/2016 2:19:12 PMI0627 21:19:12.867099       1 event.go:211] Event(api.ObjectReference{Kind:"ReplicationController", Namespace:"default", Name:"k8testrc1", UID:"9841eca3-3caa-11e6-a0d5-02e309181521", APIVersion:"v1", ResourceVersion:"164", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: k8testrc1-pu4xc
6/27/2016 2:19:12 PMI0627 21:19:12.870594       1 event.go:211] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"k8testrc1-pnxp2", UID:"984376b5-3caa-11e6-a0d5-02e309181521", APIVersion:"v1", ResourceVersion:"294", FieldPath:""}): type: 'Normal' reason: 'NodeControllerEviction' Marking for deletion Pod k8testrc1-pnxp2 from Node soumyamasterkubeenvhost1
6/27/2016 2:19:17 PMI0627 21:19:17.918805       1 event.go:211] Event(api.ObjectReference{Kind:"Node", Namespace:"", Name:"soumyamasterkubeenvhost2", UID:"soumyamasterkubeenvhost2", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RemovingNode' Node soumyamasterkubeenvhost2 event: Removing Node soumyamasterkubeenvhost2 from NodeController
6/27/2016 2:19:17 PMI0627 21:19:17.918885       1 event.go:211] Event(api.ObjectReference{Kind:"Node", Namespace:"", Name:"soumyamasterkubeenvhost1", UID:"soumyamasterkubeenvhost1", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RemovingNode' Node soumyamasterkubeenvhost1 event: Removing Node soumyamasterkubeenvhost1 from NodeController
6/27/2016 2:19:17 PME0627 21:19:17.926438       1 event.go:193] Server rejected event '&api.Event{TypeMeta:unversioned.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"soumyamasterkubeenvhost1.145c0c946160ee5a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"4", Generation:0, CreationTimestamp:unversioned.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*unversioned.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"soumyamasterkubeenvhost1", UID:"soumyamasterkubeenvhost1", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"RemovingNode", Message:"Node soumyamasterkubeenvhost1 event: Removing Node soumyamasterkubeenvhost1 from NodeController", Source:api.EventSource{Component:"controllermanager", Host:""}, FirstTimestamp:unversioned.Time{Time:time.Time{sec:63602658210, nsec:0, loc:(*time.Location)(0x27c2ee0)}}, LastTimestamp:unversioned.Time{Time:time.Time{sec:63602659157, nsec:918771383, loc:(*time.Location)(0x27c2ee0)}}, Count:2, Type:"Normal"}': 'events "soumyamasterkubeenvhost1.145c0c946160ee5a" not found' (will not retry!)
6/27/2016 2:21:03 PMW0627 21:21:03.797749       1 request.go:344] Field selector: v1 - serviceaccounts - metadata.name - default: need to check if this is versioned correctly.
6/27/2016 2:21:38 PMI0627 21:21:38.222995       1 event.go:211] Event(api.ObjectReference{Kind:"Node", Namespace:"", Name:"soumyamasterkubeenvhost1", UID:"soumyamasterkubeenvhost1", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node soumyamasterkubeenvhost1 event: Registered Node soumyamasterkubeenvhost1 in NodeController
6/27/2016 2:21:38 PME0627 21:21:38.225890       1 event.go:193] Server rejected event '&api.Event{TypeMeta:unversioned.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"soumyamasterkubeenvhost1.145c0ba78ceb87a4", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"167", Generation:0, CreationTimestamp:unversioned.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*unversioned.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"soumyamasterkubeenvhost1", UID:"soumyamasterkubeenvhost1", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"RegisteredNode", Message:"Node soumyamasterkubeenvhost1 event: Registered Node soumyamasterkubeenvhost1 in NodeController", Source:api.EventSource{Component:"controllermanager", Host:""}, FirstTimestamp:unversioned.Time{Time:time.Time{sec:63602657192, nsec:0, loc:(*time.Location)(0x27c2ee0)}}, LastTimestamp:unversioned.Time{Time:time.Time{sec:63602659298, nsec:222865241, loc:(*time.Location)(0x27c2ee0)}}, Count:3, Type:"Normal"}': 'events "soumyamasterkubeenvhost1.145c0ba78ceb87a4" not found' (will not retry!)
6/27/2016 2:22:58 PMI0627 21:22:58.486792       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a9ad257d83cab11e6a0d502e30918152] [Region1]
6/27/2016 2:22:58 PMI0627 21:22:58.488526       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a982eaaaf3caa11e6a0d502e30918152] [Region1]
6/27/2016 2:22:58 PMI0627 21:22:58.491324       1 servicecontroller.go:298] LB doesn't need update for service default/kubernetes
6/27/2016 2:22:58 PMI0627 21:22:58.531447       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:22:58 PMI0627 21:22:58.555904       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:27:01 PMW0627 21:27:01.800336       1 request.go:344] Field selector: v1 - serviceaccounts - metadata.name - default: need to check if this is versioned correctly.
6/27/2016 2:27:57 PMI0627 21:27:57.548599       1 servicecontroller.go:298] LB doesn't need update for service default/kubernetes
6/27/2016 2:27:57 PMI0627 21:27:57.554646       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a982eaaaf3caa11e6a0d502e30918152] [Region1]
6/27/2016 2:27:57 PMI0627 21:27:57.554679       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a9ad257d83cab11e6a0d502e30918152] [Region1]
6/27/2016 2:27:57 PMI0627 21:27:57.592781       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:27:57 PMI0627 21:27:57.594016       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:32:15 PME0627 21:32:15.140970       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:20 PME0627 21:32:20.180593       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:25 PME0627 21:32:25.185610       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:30 PME0627 21:32:30.190949       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:35 PME0627 21:32:35.195768       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:40 PME0627 21:32:40.201274       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:45 PME0627 21:32:45.206197       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:50 PME0627 21:32:50.269782       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:55 PME0627 21:32:55.274530       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:32:56 PMI0627 21:32:56.499458       1 servicecontroller.go:298] LB doesn't need update for service default/kubernetes
6/27/2016 2:32:56 PMI0627 21:32:56.501571       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a982eaaaf3caa11e6a0d502e30918152] [Region1]
6/27/2016 2:32:56 PMI0627 21:32:56.509100       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a9ad257d83cab11e6a0d502e30918152] [Region1]
6/27/2016 2:32:56 PMI0627 21:32:56.536381       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:32:56 PMI0627 21:32:56.542205       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:33:00 PME0627 21:33:00.320778       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:33:45 PME0627 21:33:45.461303       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:33:50 PME0627 21:33:50.505646       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:33:55 PME0627 21:33:55.510021       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:34:00 PME0627 21:34:00.514406       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:34:34 PMW0627 21:34:34.804222       1 request.go:344] Field selector: v1 - serviceaccounts - metadata.name - default: need to check if this is versioned correctly.
6/27/2016 2:35:35 PME0627 21:35:35.778797       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:35:40 PME0627 21:35:40.823169       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:35:45 PME0627 21:35:45.876936       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:35:50 PME0627 21:35:50.882134       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:36:15 PME0627 21:36:15.956173       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:36:20 PME0627 21:36:20.960586       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:36:35 PME0627 21:36:35.979712       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:36:41 PME0627 21:36:41.024454       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:36:46 PME0627 21:36:46.032171       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:36:51 PME0627 21:36:51.036356       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:37:55 PMI0627 21:37:55.506906       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a982eaaaf3caa11e6a0d502e30918152] [Region1]
6/27/2016 2:37:55 PMI0627 21:37:55.511183       1 servicecontroller.go:298] LB doesn't need update for service default/kubernetes
6/27/2016 2:37:55 PMI0627 21:37:55.512421       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a9ad257d83cab11e6a0d502e30918152] [Region1]
6/27/2016 2:37:55 PMI0627 21:37:55.542325       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:37:55 PMI0627 21:37:55.548167       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:38:06 PME0627 21:38:06.279755       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:38:11 PME0627 21:38:11.325830       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:38:26 PME0627 21:38:26.343015       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:38:31 PME0627 21:38:31.347792       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:38:36 PME0627 21:38:36.354524       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:38:41 PME0627 21:38:41.397048       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:38:46 PME0627 21:38:46.400774       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:38:51 PME0627 21:38:51.406006       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:39:16 PME0627 21:39:16.481164       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:39:21 PME0627 21:39:21.486811       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:39:36 PMI0627 21:39:36.554508       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:39:36 PMI0627 21:39:36.554627       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:39:41 PMI0627 21:39:41.632146       1 rancher.go:534] ExternalID [soumyamasterkubeenvhost1]
6/27/2016 2:39:41 PMI0627 21:39:41.632231       1 rancher.go:540] InstanceID [soumyamasterkubeenvhost1]
6/27/2016 2:40:06 PME0627 21:40:06.843398       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:40:11 PME0627 21:40:11.848007       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:40:16 PME0627 21:40:16.851739       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:40:21 PME0627 21:40:21.856012       1 nodecontroller.go:229] Error monitoring node status: an error on the server has prevented the request from succeeding
6/27/2016 2:42:54 PMI0627 21:42:54.560133       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a982eaaaf3caa11e6a0d502e30918152] [Region1]
6/27/2016 2:42:54 PMI0627 21:42:54.560154       1 servicecontroller.go:298] LB doesn't need update for service default/kubernetes
6/27/2016 2:42:54 PMI0627 21:42:54.561666       1 rancher.go:246] EnsureLoadBalancerDeleted [lb-a9ad257d83cab11e6a0d502e30918152] [Region1]
6/27/2016 2:42:54 PMI0627 21:42:54.595029       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:42:54 PMI0627 21:42:54.598638       1 rancher.go:253] Couldn't find LB %!s(MISSING) to delete. Nothing to do.
6/27/2016 2:42:55 PMW0627 21:42:55.807160       1 request.go:344] Field selector: v1 - serviceaccounts - metadata.name - default: need to check if this is versioned correctly.

@LLParse not sure what is going on here. etcd1 gets recreated while etcd2/etcd3 keep on running, yet all the information is lost. Previously created Replication controllers are no longer present in the etcd database.

@soumyalj I am unable to reproduce - shutting off one host causes etcd to reschedule to the 4th host and all k8s data is preserved.
bash-4.3# etcdctl cluster-health
member 95d0726fdbaf025 is healthy: got healthy result from http://<IP_ADDRESS>:2
379
member bb411fc35bf55e57 is healthy: got healthy result from http://<IP_ADDRESS>
:2379
member e9bb489c39c19fd8 is healthy: got healthy result from http://<IP_ADDRESS>:
2379
cluster is healthy

bash-4.3# etcdctl ls /registry/pods/default
/registry/pods/default/redis-master-lo8ey
/registry/pods/default/redis-slave-gxb3a
/registry/pods/default/frontend-mw496
/registry/pods/default/frontend-b6hqf
/registry/pods/default/frontend-f2nar
/registry/pods/default/mysql
/registry/pods/default/ghost
/registry/pods/default/cassandra-ifwuj
/registry/pods/default/cassandra-kw3ez
/registry/pods/default/cassandra-vo05c
/registry/pods/default/redis-slave-378o8

@LLParse I believe the error happens when you bring the host that used to be down, back to the pool

Done (in GCE), host came back, nothing bad happened. I SSH'd into the restarted VM fast enough to observe the old containers in 'STOPPED' state, I'm guessing where this was tested that was not the case.

Re-tested using the new qa template and observed the documented behavior. Since we added scale_policy, we can no longer rely on metadata scale value for decision making. Etcd changes imminent.

Duplicate of https://github.com/rancher/rancher/issues/5306