Core Data Relationships

Are there any plans to have this support relationships and nested entities by recursing through the JSON?

Hi @bonebox,
It already does this, take a look at the DesignerNews Example => https://github.com/hyperoslo/Sync/tree/master/Examples/DesignerNews
There we have posts that have a to-many relationship to comments. If something is not clear please let me know. :)

Hi @3lvis,
Thanks for the reply.  The first problem I had was with primary keys.  My Core Data primary keys are different than remoteID, so I added the hyper.isPrimaryKey to the respective primary key attributes on my entities, as described in the README.  However, after doing this, the app crashes.  I noticed that Sync was trying to match my entity's primary key attributes to entityName_id in the JSON.  It is just id in my JSON, so I assumed that would be automatic.  So then I also added hyper.remoteKey to the primary key and set it to id, and I can get a little further now, but it still crashes.
Now the error I'm seeing is:
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'keypath items.uniqueId not found in entity <NSSQLEntity MKItem id=1>'

items is the relationship name and uniqueId is the local primary key for MKItem.

To clarify, I have an entity called MKMarket that has the items relationship to many MKItem entities.

It turns out the issue is because my relationship is many-to-many, and apparently Sync currently only supports one-to-many.  So let me change my initial question: are there plans to support many-to-many relationships?

This doesn't seem like a bug in Sync but more like we failing on clarifying how Sync works, sorry for that.
Might be helpful if you could provide me an example of the structure of both your model and your JSON.
From what I get your JSON looks something like this:
** A list of markets:**
[
  {
    "id":"market_id",
    "other_attribute":"Market 0",
    "items":[
      {
        "id":"item_id",
        "other_attribute":"Item 0"
      }
    ]
  }
]

And your data model has a Market ---(to-many)---> Item relationship.
The primary key both on Market and Item is uniqueId.
Is this correct?

Sorry, just saw your issue, let me take a look by adding an example to our tests project.
This is A Market can have many Items and an Item can belong to more than one Market.

Yes, that's exactly right.  When I removed the many relationship from item back to market, the data gets populated successfully.  But unfortunately in the data I need to model, an item can also exist in many markets, so I need to have a many-to-many relationship between them.
FYI, I'm currently using RESTKit to populate the data and it works with the many-to-many relationship, but I'd like to switch to Sync.

Just for example, it's possible I could have something like this in the JSON:
[
  {
    "id":"1",
    "other_attribute":"Market 1",
    "items":[
      {
        "id":"1",
        "other_attribute":"Item 1"
      }
    ]
  },
    "id":"2",
    "other_attribute":"Market 2",
    "items":[
      {
        "id":"1",
        "other_attribute":"Item 1"
      }
    ]
  }
]

I took a look at our examples located in the Tests project and we have one for many-to-many.
It's a Note <--> Tag relationship, A Note has many Tags and a Tag points to many Notes.
The JSON looks like this:
[
  {
    "id": 0,
    "text": "Shawn Merril's diary, episode 0",
    "tags": [
      {
        "id": 1,
        "name": "diary"
      },
      {
        "id": 2,
        "name": "neverforget"
      }
    ]
  },
  {
    "id": 1,
    "text": "Shawn Merril's diary, episode 1",
    "tags": [
      {
        "id": 1,
        "name": "diary"
      }
    ]
  },
  {
    "id": 2,
    "text": "Shawn Merril's diary, episode 2",
    "tags": [
      {
        "id": 1,
        "name": "diary"
      }
    ]
  },
  {
    "id": 3,
    "text": "Shawn Merril's diary, episode 3",
    "tags": [
      {
        "id": 1,
        "name": "diary"
      }
    ]
  },
  {
    "id": 4,
    "text": "Shawn Merril's diary, episode 4"
  }
]

The main Core Data Model looks like this:
Model

Note

Tag

Is there anything you notice that could be off?

My model is set up very similar to that.  Are there any User Info key/value pairs that should be defined for the relationship?  How does Sync determine what the inverse relationship is?  My inverse relationship was originally called market, and I changed it to markets thinking that might be the issue, but I still get the same error.  From the error it seems as if it's looking for the item relationship in the Item entity, instead of the markets relationship.

I made a branch to look into this issue, I was able to reproduce it by adding custom local primary keys. Sync was optimized to work with a "convention over configuration" paradigm, that's why is easier to use if you follow certain "rules". A work around for your issue would be to move from uniqueId to remoteID.
I'll look closely into this issue and figure out what's wrong :)
Thanks for your helping finding this issue Jeremy, you are the best!

@3lvis thanks a lot for taking the time to work through this with me!  I totally understand the paradigm for Sync, and think it's a great idea.  I should mention that I did finally change all of my primary keys to remoteID, and in fact still get the same crash.  Now it looks like the following (only difference is reference to items.remoteID instead of items.uniqueId):
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'keypath items.remoteID not found in entity <NSSQLEntity MKItem id=1>'

FWIW, I do have a few other attributes that do not match the JSON property names, so I am using hyper.remoteKey with those, so perhaps that is still causing an issue.  Thanks again!

That's quite interesting. I made the change to remoteID and everything works fine. => https://github.com/hyperoslo/Sync/blob/fix/many-to-many-markets/Tests/Tests/Tests.m#L431-L469

I ended up changing all my other attributes to match the JSON to further simplify things, and double-checked the sample project to ensure my settings match as closely as possible, and I still get the same crash. The only difference I can think of is I used mogenerator to create my model objects, so they are subclassed.  It might still be worth looking into the crash when using custom primary keys--perhaps the fix will indirectly fix my problem as well.

@bonebox Do you have a stack trace by the way? Which method is the one that is crashing?

Here you go. It's occurring in [NSManagedObjectContext performWithOptions:andBlock] called from [DATAStack performInNewBackgroundContext] (line 250 of DATAStack.m).
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'keypath items.remoteID not found in entity <NSSQLEntity Item id=1>'
*** First throw call stack:
(
	0   CoreFoundation                      0x000000010d85dc65 __exceptionPreprocess + 165
	1   libobjc.A.dylib                     0x000000010f905bb7 objc_exception_throw + 45
	2   CoreData                            0x000000010d060bbc -[NSSQLGenerator newSQLStatementForRequest:ignoreInheritance:countOnly:nestingLevel:] + 1724
	3   CoreData                            0x000000010d04cdc4 -[NSSQLAdapter _statementForFetchRequest:ignoreInheritance:countOnly:nestingLevel:] + 244
	4   CoreData                            0x000000010cf64e0c -[NSSQLAdapter _newSelectStatementWithFetchRequest:ignoreInheritance:] + 316
	5   CoreData                            0x000000010cf64a86 -[NSSQLCore newRowsForFetchPlan:] + 118
	6   CoreData                            0x000000010cf6433c -[NSSQLCore objectsForFetchRequest:inContext:] + 524
	7   CoreData                            0x000000010cf63dbb -[NSSQLCore executeRequest:withContext:error:] + 299
	8   CoreData                            0x000000010d03ea6c __65-[NSPersistentStoreCoordinator executeRequest:withContext:error:]_block_invoke + 3356
	9   CoreData                            0x000000010d047c30 gutsOfBlockToNSPersistentStoreCoordinatorPerform + 192
	10  libdispatch.dylib                   0x00000001103ec614 _dispatch_client_callout + 8
	11  libdispatch.dylib                   0x00000001103d2002 _dispatch_barrier_sync_f_invoke + 365
	12  CoreData                            0x000000010d039245 _perform + 197
	13  CoreData                            0x000000010cf63a58 -[NSPersistentStoreCoordinator executeRequest:withContext:error:] + 504
	14  CoreData                            0x000000010cf622ca -[NSManagedObjectContext executeFetchRequest:error:] + 586
	15  DATAObjectIDs                       0x000000010cabbf17 __78+[DATAObjectIDs objectIDsInEntityNamed:withAttributesNamed:context:predicate:]_block_invoke + 599
	16  CoreData                            0x000000010cfa8a79 developerSubmittedBlockToNSManagedObjectContextPerform + 201
	17  CoreData                            0x000000010cfa8951 -[NSManagedObjectContext performBlockAndWait:] + 257
	18  DATAObjectIDs                       0x000000010cabbb0b +[DATAObjectIDs objectIDsInEntityNamed:withAttributesNamed:context:predicate:] + 491
	19  DATAFilter                          0x000000010cab620a +[DATAFilter changes:inEntityNamed:localKey:remoteKey:context:predicate:inserted:updated:] + 442
	20  Sync                                0x000000010ceb8e51 +[Sync changes:inEntityNamed:predicate:parent:inContext:dataStack:completion:] + 913
	21  Sync                                0x000000010cebaa4a -[NSManagedObject(Sync) sync_processToManyRelationship:usingDictionary:andParent:dataStack:] + 1882
	22  Sync                                0x000000010ceb9f99 -[NSManagedObject(Sync) sync_processRelationshipsUsingDictionary:andParent:dataStack:] + 537
	23  Sync                                0x000000010ceb91d8 __78+[Sync changes:inEntityNamed:predicate:parent:inContext:dataStack:completion:]_block_invoke + 152
	24  DATAFilter                          0x000000010cab67fe +[DATAFilter changes:inEntityNamed:localKey:remoteKey:context:predicate:inserted:updated:] + 1966
	25  Sync                                0x000000010ceb8e51 +[Sync changes:inEntityNamed:predicate:parent:inContext:dataStack:completion:] + 913
	26  Sync                                0x000000010ceb8379 __61+[Sync changes:inEntityNamed:predicate:dataStack:completion:]_block_invoke + 121
	27  DATAStack                           0x000000010cad076a __43-[DATAStack performInNewBackgroundContext:]_block_invoke + 58
	28  CoreData                            0x000000010cfa8a79 developerSubmittedBlockToNSManagedObjectContextPerform + 201
	29  libdispatch.dylib                   0x00000001103ec614 _dispatch_client_callout + 8
	30  libdispatch.dylib                   0x00000001103d36a7 _dispatch_queue_drain + 2176
	31  libdispatch.dylib                   0x00000001103d2cc0 _dispatch_queue_invoke + 235
	32  libdispatch.dylib                   0x00000001103d63b9 _dispatch_root_queue_drain + 1359
	33  libdispatch.dylib                   0x00000001103d7b17 _dispatch_worker_thread3 + 111
	34  libsystem_pthread.dylib             0x0000000110759637 _pthread_wqthread + 729
	35  libsystem_pthread.dylib             0x000000011075740d start_wqthread + 13
)
libc++abi.dylib: terminating with uncaught exception of type NSException

@bonebox Looks like you added remoteID to Market but not to Item, would mind checking again?

It is definitely on both. I just checked again to verify. Here's a screenshot in case you don't believe me ;)

@3lvis just FYI, I updated to the latest (1.0.3) and still get this exact same crash when doing many-to-many.

@bonebox Will have a look :)

Thanks!!! Let me know if I can provide any other data that might be helpful to you.  FYI the screenshot above is the entire model, so it's not too complex, but there are quite a few markets and items in the JSON.

Wanted to check back in on this, and provide some additional data. Not sure if it helps or not, but I decided to attempt this with an in-memory store just to see what happened, and it still crashes, but with a different error:
*** Terminating app due to uncaught exception 'NSUnknownKeyException', reason: '[<NSDictionaryMapNode 0x7fd36c48ad30> valueForUndefinedKey:]: this class is not key value coding-compliant for the key items.'
*** First throw call stack:
(
	0   CoreFoundation                      0x0000000104a25c65 __exceptionPreprocess + 165
	1   libobjc.A.dylib                     0x0000000106acdbb7 objc_exception_throw + 45
	2   CoreFoundation                      0x0000000104a258a9 -[NSException raise] + 9
	3   Foundation                          0x000000010519d82a -[NSObject(NSKeyValueCoding) valueForUndefinedKey:] + 226
	4   Foundation                          0x00000001050f6b23 -[NSObject(NSKeyValueCoding) valueForKey:] + 251
	5   CoreData                            0x0000000104193516 -[NSDictionaryMapNode valueForKey:] + 358
	6   Foundation                          0x000000010510d963 -[NSObject(NSKeyValueCoding) valueForKeyPath:] + 245
	7   Foundation                          0x0000000105131fbf -[NSFunctionExpression expressionValueWithObject:context:] + 1071
	8   Foundation                          0x0000000105131aac -[NSComparisonPredicate evaluateWithObject:substitutionVariables:] + 243
	9   CoreData                            0x0000000104191f41 -[NSDictionaryStoreMap handleFetchRequest:] + 465
	10  CoreData                            0x0000000104191251 -[NSMappedObjectStore executeFetchRequest:withContext:] + 241
	11  CoreData                            0x0000000104206a6c __65-[NSPersistentStoreCoordinator executeRequest:withContext:error:]_block_invoke + 3356
	12  CoreData                            0x000000010420fc30 gutsOfBlockToNSPersistentStoreCoordinatorPerform + 192
	13  libdispatch.dylib                   0x00000001075b4614 _dispatch_client_callout + 8
	14  libdispatch.dylib                   0x000000010759a002 _dispatch_barrier_sync_f_invoke + 365
	15  CoreData                            0x0000000104201245 _perform + 197
	16  CoreData                            0x000000010412ba58 -[NSPersistentStoreCoordinator executeRequest:withContext:error:] + 504
	17  CoreData                            0x000000010412a2ca -[NSManagedObjectContext executeFetchRequest:error:] + 586
	18  DATAObjectIDs                       0x0000000103c79f17 __78+[DATAObjectIDs objectIDsInEntityNamed:withAttributesNamed:context:predicate:]_block_invoke + 599
	19  CoreData                            0x0000000104170a79 developerSubmittedBlockToNSManagedObjectContextPerform + 201
	20  CoreData                            0x0000000104170951 -[NSManagedObjectContext performBlockAndWait:] + 257
	21  DATAObjectIDs                       0x0000000103c79b0b +[DATAObjectIDs objectIDsInEntityNamed:withAttributesNamed:context:predicate:] + 491
	22  DATAFilter                          0x0000000103c7422a +[DATAFilter changes:inEntityNamed:localKey:remoteKey:context:predicate:inserted:updated:] + 442
	23  Sync                                0x0000000104082fbc +[Sync changes:inEntityNamed:predicate:parent:inContext:dataStack:completion:] + 1484
	24  Sync                                0x0000000104081632 -[NSManagedObject(Sync) sync_processToManyRelationship:usingDictionary:andParent:dataStack:] + 1986
	25  Sync                                0x0000000104080a9e -[NSManagedObject(Sync) sync_processRelationshipsUsingDictionary:andParent:dataStack:error:] + 558
	26  Sync                                0x00000001040832ac __78+[Sync changes:inEntityNamed:predicate:parent:inContext:dataStack:completion:]_block_invoke + 172
	27  DATAFilter                          0x0000000103c74806 +[DATAFilter changes:inEntityNamed:localKey:remoteKey:context:predicate:inserted:updated:] + 1942
	28  Sync                                0x0000000104082fbc +[Sync changes:inEntityNamed:predicate:parent:inContext:dataStack:completion:] + 1484
	29  Sync                                0x0000000104082259 __61+[Sync changes:inEntityNamed:predicate:dataStack:completion:]_block_invoke + 121
	30  DATAStack                           0x0000000103c8e76a __43-[DATAStack performInNewBackgroundContext:]_block_invoke + 58
	31  CoreData                            0x0000000104170a79 developerSubmittedBlockToNSManagedObjectContextPerform + 201
	32  libdispatch.dylib                   0x00000001075b4614 _dispatch_client_callout + 8
	33  libdispatch.dylib                   0x000000010759b6a7 _dispatch_queue_drain + 2176
	34  libdispatch.dylib                   0x000000010759acc0 _dispatch_queue_invoke + 235
	35  libdispatch.dylib                   0x000000010759e3b9 _dispatch_root_queue_drain + 1359
	36  libdispatch.dylib                   0x000000010759fb17 _dispatch_worker_thread3 + 111
	37  libsystem_pthread.dylib             0x0000000107921637 _pthread_wqthread + 729
	38  libsystem_pthread.dylib             0x000000010791f40d start_wqthread + 13
)
libc++abi.dylib: terminating with uncaught exception of type NSException

@bonebox could you try this with master? Pushed a few improvements a few days ago.

Hey @3lvis, well a bit of good news--the sync works now without crashing!  However, I still get a crash when I attempt a fetch request.  Here's the predicate I'm using (this worked previously when fetching after importing to Core Data via RestKit):
request.predicate = NSPredicate(format: "ANY market.remoteID = %@", market.remoteID)
And the stack trace:
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'keypath market.remoteID not found in entity <NSSQLEntity Item id=1>'
*** First throw call stack:
(
	0   CoreFoundation                      0x000000010a8d1c65 __exceptionPreprocess + 165
	1   libobjc.A.dylib                     0x000000010c979bb7 objc_exception_throw + 45
	2   CoreData                            0x000000010a0d4bbc -[NSSQLGenerator newSQLStatementForRequest:ignoreInheritance:countOnly:nestingLevel:] + 1724
	3   CoreData                            0x000000010a0c0dc4 -[NSSQLAdapter _statementForFetchRequest:ignoreInheritance:countOnly:nestingLevel:] + 244
	4   CoreData                            0x0000000109fd8e0c -[NSSQLAdapter _newSelectStatementWithFetchRequest:ignoreInheritance:] + 316
	5   CoreData                            0x0000000109fd8a86 -[NSSQLCore newRowsForFetchPlan:] + 118
	6   CoreData                            0x0000000109fd833c -[NSSQLCore objectsForFetchRequest:inContext:] + 524
	7   CoreData                            0x0000000109fd7dbb -[NSSQLCore executeRequest:withContext:error:] + 299
	8   CoreData                            0x000000010a0b2a6c __65-[NSPersistentStoreCoordinator executeRequest:withContext:error:]_block_invoke + 3356
	9   CoreData                            0x000000010a0bbc30 gutsOfBlockToNSPersistentStoreCoordinatorPerform + 192
	10  libdispatch.dylib                   0x000000010d460614 _dispatch_client_callout + 8
	11  libdispatch.dylib                   0x000000010d446002 _dispatch_barrier_sync_f_invoke + 365
	12  CoreData                            0x000000010a0ad245 _perform + 197
	13  CoreData                            0x0000000109fd7a58 -[NSPersistentStoreCoordinator executeRequest:withContext:error:] + 504
	14  CoreData                            0x0000000109fd62ca -[NSManagedObjectContext executeFetchRequest:error:] + 586
	15  CoreData                            0x000000010a01d1c2 -[NSManagedObjectContext(_NestedContextSupport) _parentObjectsForFetchRequest:inContext:error:] + 418
	16  CoreData                            0x000000010a091c1d __82-[NSManagedObjectContext(_NestedContextSupport) executeRequest:withContext:error:]_block_invoke + 589
	17  CoreData                            0x000000010a01cfd4 internalBlockToNSManagedObjectContextPerform + 84
	18  libdispatch.dylib                   0x000000010d460614 _dispatch_client_callout + 8
	19  libdispatch.dylib                   0x000000010d446002 _dispatch_barrier_sync_f_invoke + 365
	20  CoreData                            0x000000010a01ceed _perform + 221
	21  CoreData                            0x000000010a01cd18 -[NSManagedObjectContext(_NestedContextSupport) executeRequest:withContext:error:] + 296
	22  CoreData                            0x0000000109fd62ca -[NSManagedObjectContext executeFetchRequest:error:] + 586
	23  CoreData                            0x000000010a0edace __43-[NSFetchedResultsController performFetch:]_block_invoke + 270
	24  CoreData                            0x000000010a01ca79 developerSubmittedBlockToNSManagedObjectContextPerform + 201
	25  CoreData                            0x000000010a01c951 -[NSManagedObjectContext performBlockAndWait:] + 257
	26  CoreData                            0x000000010a0ed8ac -[NSFetchedResultsController performFetch:] + 524
	27  DATASource                          0x0000000109b2b44f -[DATASource initWithFetchRequest:cellIdentifier:mainContext:] + 431
	28  DATASource                          0x0000000109b2b18e -[DATASource initWithCollectionView:fetchRequest:cellIdentifier:mainContext:configuration:] + 206
	29  IntelligentHospital                 0x00000001096e275f _TTOFCSo10DATASourcecfMS_FT14collectionViewGSQCSo16UICollectionView_12fetchRequestGSQCSo14NSFetchRequest_14cellIdentifierGSQSS_11mainContextGSQCSo22NSManagedObjectContext_13configurationGSQFTGSQPSs9AnyObject__GSQPS3___GSQCSo11NSIndexPath__T___GSQS__ + 495
	30  IntelligentHospital                 0x00000001096e0b49 _TFCSo10DATASourceCfMS_FT14collectionViewGSQCSo16UICollectionView_12fetchRequestGSQCSo14NSFetchRequest_14cellIdentifierGSQSS_11mainContextGSQCSo22NSManagedObjectContext_13configurationGSQFTGSQPSs9AnyObject__GSQPS3___GSQCSo11NSIndexPath__T___GSQS__ + 185
	31  IntelligentHospital                 0x00000001096eff1a _TFC19IntelligentHospital18ScrollingItemsCell12performFetchfS0_FT_T_ + 1402
	32  IntelligentHospital                 0x0000000109719530 _TFC19IntelligentHospital12StoreBrowser9tableViewfS0_FTCSo11UITableView21cellForRowAtIndexPathCSo11NSIndexPath_CSo15UITableViewCell + 4512
	33  IntelligentHospital                 0x000000010971979f _TToFC19IntelligentHospital12StoreBrowser9tableViewfS0_FTCSo11UITableView21cellForRowAtIndexPathCSo11NSIndexPath_CSo15UITableViewCell + 79
	34  UIKit                               0x000000010b7aea28 -[UITableView _createPreparedCellForGlobalRow:withIndexPath:willDisplay:] + 508
	35  UIKit                               0x000000010b78d248 -[UITableView _updateVisibleCellsNow:isRecursive:] + 2853
	36  UIKit                               0x000000010b7a38a9 -[UITableView layoutSubviews] + 210
	37  UIKit                               0x000000010b72da2b -[UIView(CALayerDelegate) layoutSublayersOfLayer:] + 536
	38  QuartzCore                          0x000000010b530ec2 -[CALayer layoutSublayers] + 146
	39  QuartzCore                          0x000000010b5256d6 _ZN2CA5Layer16layout_if_neededEPNS_11TransactionE + 380
	40  QuartzCore                          0x000000010b525546 _ZN2CA5Layer28layout_and_display_if_neededEPNS_11TransactionE + 24
	41  QuartzCore                          0x000000010b491886 _ZN2CA7Context18commit_transactionEPNS_11TransactionE + 242
	42  QuartzCore                          0x000000010b492a3a _ZN2CA11Transaction6commitEv + 462
	43  UIKit                               0x000000010b6aa626 _UIApplicationHandleEventQueue + 2140
	44  CoreFoundation                      0x000000010a805431 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
	45  CoreFoundation                      0x000000010a7fb2fd __CFRunLoopDoSources0 + 269
	46  CoreFoundation                      0x000000010a7fa934 __CFRunLoopRun + 868
	47  CoreFoundation                      0x000000010a7fa366 CFRunLoopRunSpecific + 470
	48  GraphicsServices                    0x000000010f6dda3e GSEventRunModal + 161
	49  UIKit                               0x000000010b6ad900 UIApplicationMain + 1282
	50  IntelligentHospital                 0x00000001096fbb57 main + 135
	51  libdyld.dylib                       0x000000010d494145 start + 1
	52  ???                                 0x0000000000000001 0x0 + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException

@bonebox Hi, found the problem in your predicate, you should use the relationship instead of the entity name. I changed the predicate from market to markets and works.
This:
request.predicate = NSPredicate(format: "ANY markets.remoteID = %@", market.remoteID)

Instead of this:
request.predicate = NSPredicate(format: "ANY market.remoteID = %@", market.remoteID)

Wow, sorry for that!  Pretty obvious, I should have caught that. My original model was one-to-many, so the original relationship was called "market", after changing it to many-to-many, I didn't bother to change it to "markets" until I was working on integrating Sync and made further changes to the model to work better with Sync.


compilation of octave interface fails

The compilation of the octave interface fails, here is the cmake config:
-- Using system's malloc
-- Could NOT find MOSEK (missing:  MOSEK_DIR MOSEK_INCLUDE_DIR MOSEK_LIBRARY)
-- A library with BLAS API found.
-- A library with LAPACK API found.
-- Could NOT find CPLEX (missing:  CPLEX_LIBRARY CPLEX_INCLUDE_DIR)
-- Found Arpack : /usr/lib/libarpack.so
-- Could NOT find ViennaCL (missing:  VIENNACL_INCLUDE_DIR) (Required is at least version "1.5.0")
-- checking for one of the modules 'libColPack>=1.0.9;ColPack>=1.0.9'
-- Could NOT find NLOPT (missing:  NLOPT_LIBRARY NLOPT_INCLUDE_DIR)
-- Could NOT find LPSOLVE (missing:  LPSOLVE_LIBRARIES LPSOLVE_INCLUDE_DIR)
-- Could NOT find ColPack (missing:  COLPACK_LIBRARIES COLPACK_LIBRARY_DIR COLPACK_INCLUDE_DIR)
-- Could NOT find ARPREC (missing:  ARPREC_LIBRARIES ARPREC_INCLUDE_DIR)
-- checking for one of the modules 'libjson>=0.11;json>=0.11;json-c>=0.11'
-- Could NOT find SNAPPY (missing:  SNAPPY_LIBRARIES SNAPPY_INCLUDE_DIR)
-- Lzo includes and libraries NOT found.
-- Spinlock support found
-- Supported backend(s) for linear algebra FOUND and ENABLED
-- Found Octave:      /usr/bin/octave
-- Octave-version:    4.0.2
-- Octave-API:        50
-- Octave-installdir: /usr/lib/x86_64-linux-gnu/octave/site/oct/api-v50+/x86_64-pc-linux-gnu
/home/me/progs/shogun/examples/example-generation/types/get_type_list.sh: 1: /home/me/progs/shogun/examples/example-generation/types/get_type_list.sh: Bad substitution
grep: /home/me/progs/shogun/build/../../../src/interfaces/modular/: No such file or directory
Generating LALR tables
Generating LALR tables
Generating LALR tables
-- ===================================================================================================================
-- Summary of Configuration Variables
--
-- The following OPTIONAL packages have been found:

 * CCache
 * GDB
 * OpenMP
 * BLAS
 * LAPACK
 * Atlas
 * GLPK
 * ARPACK
 * Eigen3 (required version >= 3.1.2)
 * Doxygen (required version >= 1.8.6)
 * LibXml2
 * HDF5
 * CURL
 * ZLIB
 * BZip2
 * LibLZMA
 * Spinlock
 * Threads
 * Protobuf
 * PLY

-- The following REQUIRED packages have been found:

 * SWIG (required version >= 2.0.4)
 * OpenCL
 * PythonLibs
 * PythonInterp
 * NumPy
 * Octave (required version >= 3.6)
 * R

-- The following OPTIONAL packages have not been found:

 * Mosek
 * CPLEX
 * ViennaCL (required version >= 1.5.0)
 * NLopt
 * LpSolve
 * ColPack
 * ARPREC
 * SNAPPY
 * LZO

-- ===================================================================================================================
-- Enabled Integration
--   OpenCV Integration is OFF 	 - enable with -DOpenCV=ON
-- ===================================================================================================================
-- Enabled Interfaces
--   libshogun is ON
--   python modular is ON
--   octave modular is ON
--   java modular is OFF 	 - enable with -DJavaModular=ON
--   perl modular is OFF 	 - enable with -DPerlModular=ON
--   ruby modular is OFF 	 - enable with -DRubyModular=ON
--   csharp modular is OFF 	 - enable with -DCSharpModular=ON
--   R modular is OFF 		 - enable with -DRModular=ON
--   lua modular is OFF 	 - enable with -DLuaModular=ON
--
-- Enabled legacy interfaces
--   cmdline static is OFF 	 - enable with -DCmdLineStatic=ON
--   python static is OFF 	 - enable with -DPythonStatic=ON
--   octave static is OFF 	 - enable with -DOctaveStatic=ON
--   matlab static is OFF 	 - enable with -DMatlabStatic=ON
--   R static is ON
-- ===================================================================================================================
-- To compile shogun type
--   make
--
-- To install shogun to /usr/local type
--   make install
--
-- or to install to a custom directory
--   make install DESTDIR=/my/special/path
--   (or rerun cmake with -DCMAKE_INSTALL_PREFIX=/my/special/path) to just change the prefix
-- ===================================================================================================================
-- Configuring done
-- Generating done
-- Build files have been written to: /home/me/progs/shogun/build
- Using system's malloc
-- Could NOT find MOSEK (missing:  MOSEK_DIR MOSEK_INCLUDE_DIR MOSEK_LIBRARY)
-- A library with BLAS API found.
-- A library with LAPACK API found.
-- Could NOT find CPLEX (missing:  CPLEX_LIBRARY CPLEX_INCLUDE_DIR)
-- Found Arpack : /usr/lib/libarpack.so
-- Could NOT find ViennaCL (missing:  VIENNACL_INCLUDE_DIR) (Required is at least version "1.5.0")
-- checking for one of the modules 'libColPack>=1.0.9;ColPack>=1.0.9'
-- Could NOT find NLOPT (missing:  NLOPT_LIBRARY NLOPT_INCLUDE_DIR)
-- Could NOT find LPSOLVE (missing:  LPSOLVE_LIBRARIES LPSOLVE_INCLUDE_DIR)
-- Could NOT find ColPack (missing:  COLPACK_LIBRARIES COLPACK_LIBRARY_DIR COLPACK_INCLUDE_DIR)
-- Could NOT find ARPREC (missing:  ARPREC_LIBRARIES ARPREC_INCLUDE_DIR)
-- checking for one of the modules 'libjson>=0.11;json>=0.11;json-c>=0.11'
-- Could NOT find SNAPPY (missing:  SNAPPY_LIBRARIES SNAPPY_INCLUDE_DIR)
-- Lzo includes and libraries NOT found.
-- Spinlock support found
-- Supported backend(s) for linear algebra FOUND and ENABLED
-- Found Octave:      /usr/bin/octave
-- Octave-version:    4.0.2
-- Octave-API:        50
-- Octave-installdir: /usr/lib/x86_64-linux-gnu/octave/site/oct/api-v50+/x86_64-pc-linux-gnu
/home/me/progs/shogun/examples/example-generation/types/get_type_list.sh: 1: /home/me/progs/shogun/examples/example-generation/types/get_type_list.sh: Bad substitution
grep: /home/me/progs/shogun/build/../../../src/interfaces/modular/: No such file or directory
Generating LALR tables
Generating LALR tables
Generating LALR tables
-- ===================================================================================================================
-- Summary of Configuration Variables
--
-- The following OPTIONAL packages have been found:

 * CCache
 * GDB
 * OpenMP
 * BLAS
 * LAPACK
 * Atlas
 * GLPK
 * ARPACK
 * Eigen3 (required version >= 3.1.2)
 * Doxygen (required version >= 1.8.6)
 * LibXml2
 * HDF5
 * CURL
 * ZLIB
 * BZip2
 * LibLZMA
 * Spinlock
 * Threads
 * Protobuf
 * PLY

-- The following REQUIRED packages have been found:

 * SWIG (required version >= 2.0.4)
 * OpenCL
 * PythonLibs
 * PythonInterp
 * NumPy
 * Octave (required version >= 3.6)
 * R

-- The following OPTIONAL packages have not been found:

 * Mosek
 * CPLEX
 * ViennaCL (required version >= 1.5.0)
 * NLopt
 * LpSolve
 * ColPack
 * ARPREC
 * SNAPPY
 * LZO

-- ===================================================================================================================
-- Enabled Integration
--   OpenCV Integration is OFF 	 - enable with -DOpenCV=ON
-- ===================================================================================================================
-- Enabled Interfaces
--   libshogun is ON
--   python modular is ON
--   octave modular is ON
--   java modular is OFF 	 - enable with -DJavaModular=ON
--   perl modular is OFF 	 - enable with -DPerlModular=ON
--   ruby modular is OFF 	 - enable with -DRubyModular=ON
--   csharp modular is OFF 	 - enable with -DCSharpModular=ON
--   R modular is OFF 		 - enable with -DRModular=ON
--   lua modular is OFF 	 - enable with -DLuaModular=ON
--
-- Enabled legacy interfaces
--   cmdline static is OFF 	 - enable with -DCmdLineStatic=ON
--   python static is OFF 	 - enable with -DPythonStatic=ON
--   octave static is OFF 	 - enable with -DOctaveStatic=ON
--   matlab static is OFF 	 - enable with -DMatlabStatic=ON
--   R static is ON
-- ===================================================================================================================
-- To compile shogun type
--   make
--
-- To install shogun to /usr/local type
--   make install
--
-- or to install to a custom directory
--   make install DESTDIR=/my/special/path
--   (or rerun cmake with -DCMAKE_INSTALL_PREFIX=/my/special/path) to just change the prefix
-- ===================================================================================================================
-- Configuring done
-- Generating done
-- Build files have been written to: /home/me/progs/shogun/build

And here is the error:
[ 86%] Building CXX object src/interfaces/octave_modular/CMakeFiles/octave_modular.dir/modshogunOCTAVE_wrap.cxx.o
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1611:18: error: ‘hid_t’ has not been declared
       save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats) {
                  ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1616:18: error: ‘hid_t’ has not been declared
       load_hdf5 (hid_t loc_id, const char *name, bool have_h5giterate_bug) {
                  ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1832:18: error: ‘hid_t’ has not been declared
       save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats)
                  ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1836:18: error: ‘hid_t’ has not been declared
       load_hdf5 (hid_t loc_id, const char *name, bool have_h5giterate_bug)
                  ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1850:5: error: ‘DECLARE_OCTAVE_ALLOCATOR’ does not name a type
     DECLARE_OCTAVE_ALLOCATOR;
     ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1853:43: error: expected constructor, destructor, or type conversion before ‘;’ token
   DEFINE_OCTAVE_ALLOCATOR(octave_swig_ref);
                                           ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1910:18: error: ‘hid_t’ has not been declared
       save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats) {
                  ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1915:18: error: ‘hid_t’ has not been declared
       load_hdf5 (hid_t loc_id, const char *name, bool have_h5giterate_bug) {
                  ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1921:5: error: ‘DECLARE_OCTAVE_ALLOCATOR’ does not name a type
     DECLARE_OCTAVE_ALLOCATOR;
     ^
/home/me/progs/shogun/build/src/interfaces/octave_modular/modshogunOCTAVE_wrap.cxx:1924:46: error: expected constructor, destructor, or type conversion before ‘;’ token
   DEFINE_OCTAVE_ALLOCATOR(octave_swig_packed);
                                              ^
make[2]: *** [src/interfaces/octave_modular/CMakeFiles/octave_modular.dir/modshogunOCTAVE_wrap.cxx.o] Error 1
make[1]: *** [src/interfaces/octave_modular/CMakeFiles/octave_modular.dir/all] Error 2
make: *** [all] Error 2

could you please provide the OS and octave version you are using in this case?

that was ubuntu 14.04 with octave from ppa:octave/stable v 4.0.2.
I will not be able to try things out, because I upgraded to Ubuntu 16.04 since. I am trying to compile it, but currently I have some trouble with OpenCV.

You need to build octave from source with a patch. Ubuntu's binary (including ppa) is not built with the patch so it won't work . See my post on June 4 2015 here https://github.com/shogun-toolbox/shogun/issues/2850

Thanks for clarifying @beew !


GUACAMOLE-1628: Docker image of guacamole client is missing unzip

Installs unzip and fixes docker start.sh to support setting LOGBACK_LEVEL to file path without delimiter problems

My mistake. I assumed the LOGBACK_LEVEL values were a file path. Regardless, values containing forward slashes had them parsed out as delimiters. Changing the sed delimiter to semicolons should prevent future conflicts

Changing the sed delimiter to semicolons should prevent future conflicts

Sure, except with semicolons and backslashes, as well as any character that will be interpreted by the XML parser like an ampersand or a quote.

It is quite weird that this change did not make it into neither the Dockerfile for the 1.5.0, nor 1.5.1-RC1 docker containers that are available on Dockerhub. When trying to start these containers, I get an error that unzip is not found.
In the Dockerfile:
RUN apt-get update -qq \
    && apt-get install -y xmlstarlet unzip\
    && rm -rf /var/lib/apt/lists/*

The change is correct in the Master branch, but not in the staging/1.5.1 branch
But, this apt-get is not yet merged into the branch stage/1.5.1.
It is also not available in the tag 1.5.1-RC1 (https://github.com/apache/guacamole-client/blob/1.5.1-RC1/Dockerfile)

That it did not make it into 1.5.0, I can understand. It was probably a mistake that could easily happen, but the change is correct in the master branch. I am quite surprised, though, that nobody tried to run the docker container for a month (the one here, which is broken https://hub.docker.com/layers/guacamole/guacamole/1.5.0/images/sha256-b5e3e65ce76c6f33d12fe652133c1b945a2431bafd8ae0d7eb3c5c5883e95d2d?context=explore)
I suggest that we cherry pick the Dockerfile from master, putting it into at least the staging/1.5.1-RC1 branch, or to create a branch staging-<IP_ADDRESS>, fixing it there.

That it did not make it into 1.5.0, I can understand. It was probably a mistake that could easily happen, but the change is correct in the master branch. I am quite surprised, though, that nobody tried to run the docker container for a month (the one here, which is broken
@maglub It was not a mistake - the issue and fix were out of scope for 1.5.0 and went in already tagged for 1.6.0, and we did not pull it back into the 1.5.1 release. I'd like to wrap up the 1.6.0 release, soon, anyway, so hopefully it'll be out relatively soon - or we could consider pulling a couple of fixes into a 1.5.2 release. @mike-jumper ?
Also, it isn't that the container is broken, or that no one tried to run 1.5.0 or 1.5.1. It's that it is only required if you actually try to change the LOGBACK_LEVEL variable, which is only required if you're debugging a problem with the container.

I also just came to that conclusion, looking through the startup script. Our home built helm chart is setting the LOGBACK_LEVEL. Thanks for a quick answer.


Release Google.Cloud.GdcHardwareManagement.V1Alpha version 1.0.0-alpha03

Changes in this release:
New features

Add a DeleteSite method (commit 546e30b)
Add MAC address and disk info to the Hardware resource (commit 546e30b)

Documentation improvements

Annotate rack_location field as required; this was always enforced (commit 546e30b)

Created release for Google.Cloud.GdcHardwareManagement.V1Alpha-1.0.0-alpha03


Stopped working after upgrading to Android v13

After successful login returns to login screen.
The console keeps showing message: E/FrameEvents(28003): updateAcquireFence: Did not find frame.

did you manage to solve the problem?

Works with the latest flutter_thingsboard_pe_app


handleSymbol BuildContext:

i could be doing something wrong here, so pls fire way.
it looks like h.init.* is not defined so it errs.

--> request #1: textDocument/documentSymbol: {"textDocument":{"uri":"file:///Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/handler.go"}}
langserver-go: handleSymbol - bctx: &{GOARCH:amd64 GOOS:darwin GOROOT:/usr/local/opt/go/libexec GOPATH:/Users/mbana/go CgoEnabled:true UseAllFiles:false Compiler:gc BuildTags:[] ReleaseTags:[go1.1 go1.2 go1.3 go1.4 go1.5 go1.6 go1.7] InstallSuffix: JoinPath:<nil> SplitPathList:<nil> IsAbsPath:<nil> IsDir:0xdcf20 HasSubdir:0xdcfc0 ReadDir:0xdd0c0 OpenFile:0xdce40}
langserver-go: handleSymbol - h.init: &{InitializeParams:{ProcessID:0 RootPath:file:///Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver InitializationOptions:0xc4201311d0 Capabilities:{XFilesProvider:false XContentProvider:false}} NoOSFileSystemAccess:false BuildContext:<nil> RootImportPath:github.com/sourcegraph/go-langserver/langserver}
langserver-go: Handle err - ctx: 0xc4201098e0 conn: 0xc4201098f0, req: 0xc420106820, err: unexpected panic: runtime error: invalid memory address or nil pointer dereference
panic serving textDocument/documentSymbol: runtime error: invalid memory address or nil pointer dereference
goroutine 21 [running]:
github.com/sourcegraph/go-langserver/langserver.(*LangHandler).Handle.func1(0xc4200499e0, 0xc4201098e0, 0xc4201098f0, 0xc420106820)
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/handler.go:112 +0x20e
panic(0x49cd40, 0xc4200120b0)
	/usr/local/opt/go/libexec/src/runtime/panic.go:458 +0x243
github.com/sourcegraph/go-langserver/langserver.(*LangHandler).handleSymbol(0xc420140120, 0x738940, 0xc420158690, 0x7312c0, 0xc420137490, 0xc420106820, 0x0, 0x0, 0x0, 0xc42011acc7, ...)
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/symbol.go:302 +0xa09
github.com/sourcegraph/go-langserver/langserver.(*LangHandler).handleTextDocumentSymbol(0xc420140120, 0x738940, 0xc420158690, 0x7312c0, 0xc420137490, 0xc420106820, 0xc42011acc0, 0x55, 0x774850, 0x738940, ...)
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/symbol.go:263 +0x128
github.com/sourcegraph/go-langserver/langserver.(*LangHandler).Handle(0xc420140120, 0x7388c0, 0xc4200122b0, 0x7312c0, 0xc420137490, 0xc420106820, 0x0, 0x0, 0x730f00, 0xc42019a110)
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/handler.go:275 +0x2237
github.com/sourcegraph/go-langserver/langserver.(*LangHandler).handle(0xc420140120, 0x7388c0, 0xc4200122b0, 0xc420137490, 0xc420106820, 0x70, 0x0, 0xc420112340, 0x563b60)
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/handler.go:94 +0x66
github.com/sourcegraph/go-langserver/langserver.(*LangHandler).(github.com/sourcegraph/go-langserver/langserver.handle)-fm(0x7388c0, 0xc4200122b0, 0xc420137490, 0xc420106820, 0xc420049b08, 0x49e400, 0xc420109760, 0x16)
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/handler.go:29 +0x52
github.com/sourcegraph/jsonrpc2.HandlerWithError.Handle(0xc420108700, 0x7388c0, 0xc4200122b0, 0xc420137490, 0xc420106820)
	/Users/mbana/go/src/github.com/sourcegraph/jsonrpc2/handler_with_error.go:15 +0x6f
github.com/sourcegraph/jsonrpc2.(*Conn).readMessages(0xc420137490, 0x7388c0, 0xc4200122b0, 0xc42011ac00)
	/Users/mbana/go/src/github.com/sourcegraph/jsonrpc2/jsonrpc2.go:473 +0x81b
github.com/sourcegraph/jsonrpc2.NewConn(0x7388c0, 0xc4200122b0, 0x738100, 0xc4201584b0, 0x732d80, 0xc420108700, 0xc42010c040, 0x1, 0x1, 0xc420137340)
	/Users/mbana/go/src/github.com/sourcegraph/jsonrpc2/jsonrpc2.go:295 +0x2e6
github.com/sourcegraph/go-langserver/langserver/modes.webSocketHandler(0x737cc0, 0xc420114750, 0xc42014a000, 0xc420138240, 0xc42010c040, 0x1, 0x1)
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/modes/websocket.go:71 +0x253
created by github.com/sourcegraph/go-langserver/langserver/modes.WebSocket.func1
	/Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/modes/websocket.go:42 +0x343
<-- error #1: textDocument/documentSymbol: {"code":0,"message":"unexpected panic: runtime error: invalid memory address or nil pointer dereference","data":null}
--> request #2: textDocument/hover: {"textDocument":{"uri":"file:///Users/mbana/go/src/github.com/sourcegraph/go-langserver/langserver/handler.go"},"position":{"line":32,"character":29}}
<-- result #2: textDocument/hover: {}
warning: failed to send diagnostics: jsonrpc2: connection is closed.
^C

Did you do an init?

I actually thought we had that init check, may have regressed on that.

Funnily enough, I think we do. This code has evolved pretty badly though, so may not actually work :P https://sourcegraph.com/github.com/sourcegraph/go-langserver/-/blob/langserver/handler.go#L116-117

as an aside, are you folks happy with putting the fields into initializationOptions, https://github.com/Microsoft/vscode-languageserver-node/blob/master/client/src/main.ts#L1113, i've done this https://github.com/mbana/monaco-go/blob/master/src/language-client/monaco-language-client.ts#L27 but it's pending a push.

Yeah that makes sense to do, I believe that field didn't exist when we first created this server.

@slimsag so I think the reason this is panicing may have more to do with us having strange expectations about what is sent over init (ie accidental dependency on how sg prod does things).
Other than that, does the code look alright? My guess is yes.

personally, if i were consuming the langserver, i'd find something like below better than the panic:

do init without, say, the opts set incorrectly to handle #2 below.
try to do textDocument/documentSymbol.
validate then, if invalid, error with why.

that said, i think, the original commit was fine—h.defaultBuildContext() acts as an alias—so if PathHasPrefix(filePkgPath, bctx.GOROOT) { should work as usual.
h.defaultBuildContext() selects either the override provided, or in this case, provides a default-constructed version.


Does LEAP support genetic programming?

My question is in the title. After a quick search on the documentation page with the keywords "genetic programming", there does not seem to be any support for GP, which seems to be confirmed by the Future Work section of the associated paper. From here, it seems that now some version of LEAP supports Cartesian genetic programming. Not sure what it really is, because I've never used it.

Hi @nbro, thanks for the question!
You're correct—the development branch of LEAP now has a CGP implementation, which is the first variant of GP we've tackled.
We've been giving a lot of thought to how to make sure LEAP can support other kinds of GP seamlessly (in particular, we've been comparing different design ideas for how to best facilitate parsimony pressure, which is a must-have for many forms of GP).  But so far CGP is the only one we have code for.
(Personally, I rather like CGP, since it's resistant to bloat and pretty simple conceptually; but that's just my opinion)

@SigmaX One of the big limitations of current EC libraries is that they only support one implementation of GP, so they do not allow for the comparison of different implementations/versions of GP. It would really be nice to have a library that implements the most common versions of GP, i.e. tree-based, stack-based, CGP, etc. LEAP seems to be the newest serious player in the game, so it could support these different implementations. I'm not sure I will have the time to contribute to LEAP, but I will see in the future.

Thanks @nbro --- agreed.  The issues you raise are a big part of why we built LEAP.  We hope it can grow into a more general and flexible framework, supporting many paradigms and representations while keeping things simple.
Our impression has also been that many frameworks write themselves into a corner, so to speak, ultimately making it hard to do new things with them.  It takes quite a bit of peer review and team diversity to keep that from happening!
Right now "GP" is vaguely in our roadmap, once someone has bandwidth to work on it.  Having CGP is a start, but getting some basic tree-based GP is really the important box to check, since many people treat is as canonical.  And yes, after that something like PushGP and/or grammatical evolution would round it out.

...
Right now "GP" is vaguely in our roadmap, once someone has bandwidth to work on it. Having CGP is a start, but getting some basic tree-based GP is really the important box to check, since many people treat is as canonical. And yes, after that something like PushGP and/or grammatical evolution would round it out.

"Bandwidth" is the critical issue since it's pretty much the three of us working on this when we have the time at work.  Worse yet, development has been needs driven for @SigmaX and myself; i.e., we prioritize design implement functionality for which we have an immediate need for work projects.  However, as Siggy stated earlier, we do try for an open framework such that is easier to extend to accommodate various EC approaches.
If you're interested, we do have a Slack workspace, leap-gmu.slack.com, in which we discuss LEAP design and implementation topics; we also have a weekly scrum on Thursdays at 4pm, EST, modulo holidays, of course.  (So no meeting today! ;) ) I share the meeting link in Slack on Thursdays, but I can also add you to the formal invite that maintains the Google Meet link.
Speaking of tree representations, I think that might become a priority for version 0.6.0 because graph-based NNs are becoming popular at work, and LEAP is on the hook to support that in some way.  Naturally, once support for that is added, then tree-based GP becomes tenable.


npm package seems to be broken

I would expect that following the Readme and running
npm i -g @grnsft/if
npm i -g @grnsft/if-unofficial-models

and the following manifest file below (from co2js readme) with
impact-engine --impl path/to/manifest
should work. But it does run into an error:
... Co2jsModel is not constructable or does not belong to given plugin ....
The manifest file:
name: co2js-demo
description: example impl invoking CO2.JS model
tags:
initialize:
  models:
    - name: co2js
      model: Co2jsModel
      path: '@grnsft/if-unofficial-models'
graph:
  children:
    child:
      pipeline:
        - co2js
      config:
        co2js:
          type: swd
      inputs:
        - timestamp: 2023-07-06T00:00 # [KEYWORD] [NO-SUBFIELDS] time when measurement occurred
          duration: 1
          bytes: 1000000
          green-web-host: true
          options:
            dataReloadRatio: 0.6,
            firstVisitPercentage: 0.9,
            returnVisitPercentage: 0.1,
            gridIntensity:
              device: 560.98
              dataCenter:
                country: 'TWN'

It is not broken on main though. If I

clone the if-unofficial-models repo
npm install
npm link
clone the if repo
npm install
npm link @grnsft/if-unofficial-models

and run the manifest via npm run impact-engine -- --impl path/to/manifest it works.

This should be fixed with the latest release - please ping us if you keep having issues.


npm install parse-server has different package.json from master repo

following modules cannot be found in the package.json after npm install parse-server
"parse-server-fs-adapter": "^1.0.0",
"parse-server-gcs-adapter": "^1.0.0",
"parse-server-push-adapter": "^1.0.0",
"parse-server-s3-adapter": "^1.0.0",
"parse-server-simple-mailgun-adapter": "^1.0.0",

For implementation related questions or technical support, please refer to the Stack Overflow and Server Fault communities.
Make sure these boxes are checked before submitting your issue -- thanks for reporting issues back to Parse Server!

[x] You've met the prerequisites.

[x] You're running the latest version of Parse Server.

[x] You've searched through existing issues. Chances are that your issue has been reported or resolved before.

Environment Setup
Steps to reproduce
Logs/Trace

Of course it does, the master repo is not necessarily released on npm


Plugin installation was unsuccessful

/opt/kibana # bin/kibana plugin --install metric-vis-colors -u file:///opt/kibana/src/plugins/kibana_metric_vis_colors.zip
Installing metric-vis-colors
Attempting to transfer from file:///opt/kibana/src/plugins/kibana_metric_vis_colors.zip
Transferring 38433 bytes....................
Transfer complete
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation was unsuccessful due to error "Config schema already has key: metric_vis_colors"
/opt/kibana # ll installedPlugins
total 48
drwxr-xr-x 12  501 staff 4096 Jun 19 12:55 ./
drwxr-xr-x 10  501 staff 4096 May  5 01:16 ../
drwxr-xr-x  5 root root  4096 May  5 02:39 graph/
drwxr-xr-x  3 root root  4096 May  5 02:33 heatmap/
drwxr-xr-x  7 root root  4096 May  5 03:00 kbn_sankey_vis/
drwxr-xr-x  6 root root  4096 May  5 02:33 sense/
drwxr-xr-x  3 root root  4096 May  5 02:33 tagcloud/
drwxr-xr-x 10 root root  4096 May  5 02:33 timelion/
drwxr-xr-x  3 root root  4096 May  5 02:33 vectormap/
===>root – <IP_ADDRESS> – /opt/kibana\

I am getting the same too...
Plugin installation was unsuccessful due to error "Config schema already has key: metric_vis_colors"
I tried manually removing the enty from some ./optimize/.babelcache.json file too.
Nothing helped.

Which version of Kibana are you using ?

4.5.1
kibana-4.5.1-linux-x64

FATAL [Error: Config schema already has key: metric_vis_colors]
{"type":"log","@timestamp":"2016-06-24T05:37:36+00:00","tags":["fatal"],"pid":1864,"level":"fatal","message":"Config schema already has key: metric_vis_colors","error":{"message":"Config schema already has key: metric_vis_colors","name":"Error","stack":"Error: Config schema already has key: metric_vis_colors\n    at Config.extendSchema (/home/aksbhatia/elastic/kibana-4.5.1-linux-x64/src/server/config/config.js:61:15)\n    at ScopedPlugin.readConfig$ (/home/aksbhatia/elastic/kibana-4.5.1-linux-x64/src/server/plugins/Plugin.js:68:20)\n    at tryCatch (/home/aksbhatia/elastic/kibana-4.5.1-linux-x64/node_modules/babel-runtime/regenerator/runtime.js:67:40)\n

@valouille did you check on this?

Hi all,
Same here, i can't install the plugin

Someone know what to do to overcome ?
Thx all

Hello,
The problem was the installation process. We should not clone the repository in $KIBANA_HOME/src/plugins.
So the fix is to remove it from this directory, and install it from another:
rm -rf $KIBANA_HOME/src/plugins/kibana_metric_vis_colors*
cd ~
git clone https://github.com/Oxalide/kibana_metric_vis_colors.git
zip -r kibana_metric_vis_colors kibana_metric_vis_colors
$KIBANA_HOME/bin/kibana plugin --install metric_vis_color -u file:///$HOME_DIR/kibana_metric_vis_colors.zip

I will update the installation process
VaLouille


Add info about passthrough-motd issues

Currently there is an issue where if your MOTD is either too long, or uses special characters, that the Bedrock client cannot connect. This adds instructions for users to try disabling this setting to see if it helps.
My MOTD was extremely long and had both emojis and a lot of color formatting. When I disabled passthrough-motd I was able to successfully connect!

Thank you for the contribution! Sorry to hear you had difficulties connecting.
I would rather if we find out what the specific limitations are for Bedrock clients. If we know what characters make Bedrock freak out, then we should just strip these characters within Geyser instead of having users worry about it.
But if we can't figure that out then this works as a temporary measure.

I would just restrict it to ASCII and the color codes bedrock supports... bedrock will probably throw a fit about much more than that.

Those are both great suggestions! If I have some time I will try to do some testing to see if I can figure out what characters specifically are having troubles. Here is the MOTD that was having the issue:
§d✨§bW§aE§dL§cC§eO§bM§aE§d✨ §6to our §dR§aA§bI§cN§dB§eO§aW §eH§dA§bV§aE§cN\! §6🌈 §f  §6💖§aGet ready for §cA§dD§bV§aE§cN§dT§bU§aR§cE, §dF§bR§aI§cE§dN§bD§aS§cH§dI§bP§a, and §eL§dO§bT§aS of §cL§dO§bV§aE§c\!§6🐾

Hey there! Thank you for contributing to the wiki. We've migrated the wiki to our new GeyserMC/GeyserWebsite. If you'd like to, please migrate your PR to the new repo. From there, we can review it. Thank you for your contributions!


alertEventsResponse has wrong json tag

Description
Recently i was trying to retrieve my alert events, no error was caused, the body was full
but in response object was no data.
Go Version
1.18
Current behavior
Response is empty
Expected behavior
The response is full of objects
Steps To Reproduce
Steps to reproduce the behavior:

Use client in go
Call ListAlertEvents
Call curl provided below
In go we have nil response - by using curl we have successfully retrieved objects.

Additional Context
Struct view at the moment:
type alertEventsResponse struct { 	AlertEvents []*AlertEvent json:"alert_events,omitempty" }
Need to change to
type alertEventsResponse struct { 	AlertEvents []*AlertEvent json:"recent_events,omitempty" }
Let's do curl
curl -X GET 'https://api.newrelic.com/v2/alerts_events.json' \      -H 'X-Api-Key:MY_KEY' -i
We retrieve
{"recent_events":[...]}
So we have the wrong json tags (in the request we get recent_events, in the client we have alert_events.

Merged fix and released. Thanks!


compiled js files are contained in test run because default testMatch has js files in glob

Issue :
When using pure TypeScript and default ts-jest preset, if there are compiled TS test files they will be executed as well and throw an error if es2015 modules are used as they are not transformed.
Culpirt is the default testMatch which has following glob:
https://github.com/kulshekhar/ts-jest/blob/master/src/config/create-jest-preset.ts#L11
{
 testMatch: ['**/__tests__/**/*.js?(x)', '**/?(*.)+(spec|test).js?(x)'],
}

Expected behavior :
default testMatch should be used only for jsWithTs and jsWithBabel presets to mitigate this behaviour.
Also if user provides outDir within tsConfig that glob should be added to testPathIgnorePatterns: [tsconfig.compileOptions.outDir]
I can submit PR if you want with fixed behaviour. Lemme know. Thanks for ts-jest ! cheers 🍻

Quick answer, I have to run sorry. Would love a PR, but:

you cannot have a dynamic preset. ie, cannot make-up some jest config options depending on the tsconfig ones. can't explain now why, no time sorry
ts-jest is to extend jest, so we just extend the default jest options in the preset, without removing any

I'd let this open for discussion and will think more about it, but in your case you'd:
module.expotrs = {
	preset: 'ts-jest',
	testMatch: ['**/__tests__/**/*.ts?(x)', '**/?(*.)+(spec|test).ts?(x)'],
}

Hey ! no worries. yes I can do that manually indeed like following to cover both cases. I just wanted to get better DX by default for everyone. But ofc what you're saying makes sense.
From logical perspective it makes sense to override that default just to include ts files because only those are processed within transform.
// @ts-check

/**
 * @type {jest.InitialOptions}
 */
const config = {
  preset: 'ts-jest',
  testPathIgnorePatterns: ['out-tsc'],
  setupTestFrameworkScriptFile: '<rootDir>/src/setup-tests.ts'
}

module.exports = config

So should I submit PR for removal of default testMatch ?

@Hotell I'll suggest you to discuss about that with @kulshekhar as well, maybe on slack and then shorten the story here. I totally understand your point of view, but as you'd normally not compile test files, there should be no ts test files that should become js files. Also following your logic we should also remove js-related stuff from moduleFileExtensions, and maybe some others related to js.
 This is just the defaults as jest defines it, except in jest 22 there is no such export so it has been copied there

Given that this issue is caused only when test files are built along with the rest of the project, it seems the correct option will be to fix the build configuration. Updating ts-jest to account for this case would just be adding unneeded complexity here. To illustrate my point, this is the first time I've heard about such an issue.

@kulshekhar

To illustrate my point, this is the first time I've heard about such an issue.

Interesting :) I've been using TS since version 0.9 and compiling tests with source files was always a must because otherwise you cannot check if types are correct everywhere ( no matter if you have one config for everything or separate configs for app an tests ) when manually building on CI via tsc and so on. IMHO I don't see by adding one more ternary some additional complexity, but it's your code so you know what's best for you and I'm fine with that.
I'll close this as I don't see to get any agreement from your side. anyway thanks for spending at least some time on this discussion! cheers

PS: @huafu  discussing stuff on slack channel is a big no no for OSS, it's not open and information will get lost eventually :)

you cannot check if types are correct everywhere

Now typechecking is done by ts-jest

discussing stuff on slack channel is a big no no for OSS

By this I mean discussing in the chat and posting shortened version with result of discussion in here. It's ok to exchange here, tho once it's needs some quick q/a which end up in 50+ posts in a github thread, it's almost useless. You endup going to the end of the thread to try to have the final result of discussion, and it's usually never ending. But yeah, again, I agree when it can be handled in a very few posts in a GH comment thread.

Also I completely forgot to add, that while it's nice that ts-jest now handles compilation errors etc, if you're using tslint, you'll need get additional npm scripts and tsconfig file to lint your tests.  This is especially "painful" when leveraging commit hooks with lint-staged and friends, because tslint --project tsconfig.json will no longer work, even throw errors when used from lint-staged ( as we are trying to apply lint on file which is not included within tsconfig )
tslint --project tsconfig.json --format codeFrame "--fix"

"/path-to-project-folder/src/utils/functions.ts"
"/path-to-project-folder/src/__tests__/utils/functions.spec.ts"

'/path-to-project-folder/src/__tests__/utils/functions.spec.ts' is not included in project.

npm ERR! Exit status 1


Trying Sort by Script. Kibana worked but GoLang failed on JSON parsing, "_script"

Hi,
My ElasticSearch version is 6.4
and I am using olivere's client in GoLang
I am trying to sort the result by a custom script which is introduced here:
https://www.elastic.co/guide/en/elasticsearch/reference/6.4/search-request-sort.html#_script_based_sorting
Here's my GoLang search code:
func (s *ElasticSearchClient) Search(ctx context.Context,
	index string, searchType string,
	search string) (*elastic.SearchResult, error) {
	return s.es.Search(index).Type(searchType).Source(search).Do(ctx)
}

And here's my query:
 {
  "query": {
    "bool" : {
      "must" : [
        {
          "term" : { "latestEvent" : "AAA" }
        },
        {
          "range" : {
          "latestEventDate" : { "gte" : "2023-11-10T17:40:50.07Z" }
          }
        }
      ]
    }
  },
  "sort": [
    {
      "_script": {
        "type": "number",
        "order": "desc",
        "script": {
          "source": """
            def value = {some kind of calculation}
            return value;
          """,
          "params": {
            "mean": 1,
            "sd": 2,
            "currentTimestamp":<PHONE_NUMBER>141
          }
        }
      }

    }
  ],
  "_source": [
    "latestEvent",
    "latestEventDate"
  ]
}

I wrote my query like above and confirmed it worked well in Kibana.
But I started having issues within the code.
I am facing an error because _parse is not a JSON friendly key.
"elastic: Error 400 (Bad Request): [28:21] [_script] failed to parse field [script] [type=x_content_parse_exception]"

I spent days to figure out how I can fix this but there was no luck. Can someone please help me to either

forcing JSON parsing allowing underscored keys like _script and _source
find another way to sort the result by script value

Thanks in advance!

Hello @9tontruck
Unfortunately we do not provide support for Oliver's client.
If you decide to switch to the official client I can help you rewrite your query with the TypedAPI.

@Anaethelion
Thanks for your comment. I updated my ES client so my app is using "github.com/elastic/go-elasticsearch/v7/esapi"
And my search function is something like this now:
func (e ElasticSearchClient) Search(ctx context.Context,
	index string,
	searchType string,
	search map[string]interface{},
) (*esapi.Response, error) {
	jsonString, _ := json.Marshal(search)
	return e.es.Search(
		e.es.Search.WithContext(context.Background()),
		e.es.Search.WithIndex(index),
		e.es.Search.WithBody(strings.NewReader(string(jsonString))),
		e.es.Search.WithTrackTotalHits(true),
		e.es.Search.WithContext(ctx),
	)
}

But I am still having JSON parsing errors when I add _script inside of the Sort field.
ref- https://www.elastic.co/guide/en/elasticsearch/reference/6.4/search-request-sort.html#_script_based_sorting
The version of my ElasticSearch is 6.4
Would you guide me to use scrip-sort using the go-elasticsearch client?

Created a new thread as it's closed
https://github.com/elastic/go-elasticsearch/issues/764


Consider adding CalVer as an alternative to SemVer

The badge criteria recommends SemVer exclusively. Perhaps any structured and well reviewed versioning scheme satisfies the security best practices objective. In particular CalVer is used by Ubuntu and other recognizable projects.
https://calver.org/
In particular see
https://calver.org/#when-to-use-calver
For purposes of healthy discussion here are some counter-arguments:

More suggestions are less concise.
-- More suggestions and more to read makes the process more difficult without proportionate benefit.
CalVer may be most applicable to large projects and presumably most projects are small.

It's a really interesting idea, and I can see some advantages.
There's another counter-argument: CalVer doesn't handle multiple branches well, e.g., 2.X vs. 3.X branches. Thoughts?

It's a really interesting idea, and I can see some advantages.
There's another counter-argument: CalVer doesn't handle multiple branches well, e.g., 2.X vs. 3.X branches. Thoughts?

I'm not sure what you mean but let's see if this is helpful... CalVer uses Major:Minor:Micro e.g. Year:Month:Patch. So there shouldn't be any issue with version controlling security fixes if that is the concern. For example, Ubuntu LTS has fixes that get backported into older releases. Like right now 20.04 is the latest Long Term Support (LTS) release but the 2016 release has backported fixes and is listed right now as 16.04.7.

It is helpful! I guess you're right, it doesn't have problems handling branches.
The CalVer page has an interesting description of when to use CalVer:

Does your project feature a large or constantly-changing scope?

Large systems and frameworks, like Ubuntu and Twisted.
Amorphous sets of utilities, like Boltons.

Is your project time-sensitive in any way? Do other external changes drive new project releases?

Business requirements, such as Ubuntu's focus on support schedules.
Security updates, such as certifi's need to update certificates.
Political shifts, such as pytz's handling of timezone changes.

I can see adding this as a legitimate alternative. I think the next step is to write proposed text & distribute it around for comment. I can do that unless you'd prefer to take first crack at it.

CalVer works if the project includes a "micro" value (as many do). Then, if there's a branch off an older version, it's easy to determine what's going on. However, CalVer doesn't strictly require a micro value. I think if we suggest that the CalVer include a micro value then it'd also be fine. Obviously we'll need to get a larger review, but the first step is to develop proposed changed text.
===
Here's the current text of criterion version_semver:

description: It is SUGGESTED that the Semantic Versioning (SemVer) format be used for releases.

details: Other version numbering schemes, such as commit IDs (such as git commit id or mercurial changeset id) or date-based schemes like YYYYMMDD, MAY be used as version numbers, since they are unique. Some alternatives can cause problems, because users may not be able to easily  determine if they are up-to-date. SemVer may be less helpful for identifying software releases if all recipients only run the latest version (e.g., it is the code for a single website or internet service that is constantly updated via continuous delivery).

Here's some proposed alternative text to SUGGEST either SemVer or CalVer:

description: It is SUGGESTED that the Semantic Versioning (SemVer) or Calendar Versioning (CalVer) format be used for releases. It is SUGGESTED that CalVer users include a micro level value.

details: Projects should generally prefer whatever scheme is expected by their users, e.g., because it is the normal scheme used by their ecosystem. Other version numbering schemes, such as commit IDs (such as git commit id or mercurial changeset id), MAY be used as version numbers, since they are unique. Some alternatives (such as commit IDs) can cause problems as release identifiers, because users may not be able to easily determine if they are up-to-date. Schemes may be unimportant for identifying software releases if all recipients only run the latest version (e.g., it is the code for a single website or internet service that is constantly updated via continuous delivery).

Thoughts?

P.S. Identifying versions is important for security, because we want people to quickly update if they have a vulnerable version... which means that we need to help users identify (1) the version they're using and (2) the version they should be using.

I think it makes sense to expand the allowed versioning. Kubernetes and most things in the CNCF are too large to follow semver, for example.
CalVer is nice, but none of these are perfect. What about just requiring projects to use a well-documented versioning scheme, with a few of these as suggestions.

What about just requiring projects to use a well-documented versioning scheme, with a few of these as suggestions

That's already the case; this particular criterion is just SUGGESTed.  The current text suggests only SemVer.
There had been earlier discussion about using calendar dates as version numbering, as clearly some projects do that. The problem has been that with dates there was no way to clearly distinguish branches when they live simultaneously, which does happen. CalVer includes an option (micro levels) that eliminates that historical problem of dates as version ids. E.g., 2020.9.4 is different from 2021.1.1, even if they're both "current" versions.

I think SemVer should be the recommendation for SDKs and APIs, because they Major # is important in determining backwards compatibility of the clients of those SDKs or APIs. Using CalVer does not seem to make that as obvious. In addition, for SDKs and APIs, it seems that CalVer is somewhat redundant with the release date (or at least discernible from that).

Versioning schemes are definitely a big topic to bikeshed on.

The problem has been that with dates there was no way to clearly distinguish branches when they live simultaneously, which does happen

Yup, no scheme is perfect. Some seem to fit some types of projects better than others, though.

I think SemVer should be the recommendation for SDKs and APIs,

Maybe we could build on this and provide concrete recommendations of schemes based on the type of project?
One area semver falls apart IMO is when projects get larger. This is a case of https://xkcd.com/1172/. Without a very very very well defined API surface, every change can be considered a breaking change to someone, so the semantics around "breaking" begin to lose meaning. The "correct" way according to the semver spec is to just bump the major version on each release in that case, which works but takes away all semantic meaning of the versioning spec.
Shameless plug for my personal favorite versioning scheme, RealVer :)

Like many of the best jokes, RealVer is disturbingly close to the truth :-).
The goal of SUGGESTed criteria is to encourage behavior that's usually the right thing to do, but for which there are so many exceptions that it's not worth even asking for an explanation as to why something different was done. If nothing else, it's hard to change a version numbering system once set.
The proposed text already notes that CalVer tends to be preferred as projects get large; we could add "include a large number of dependencies". We could also note that SemVer tends to be preferred for SDKs and APIs.
So I tweaked the text to hopefully cover these discussions, and while I was at it, I tried to simplify the details text in general. How about this?
====
Proposed alternative text to SUGGEST either SemVer or CalVer:

description: It is SUGGESTED that the Semantic Versioning (SemVer) or Calendar Versioning (CalVer) version numbering format be used for releases. It is SUGGESTED that those who use CalVer include a micro level value.

details: Projects should generally prefer whatever format is expected by their users, e.g., because it is the normal format used by their ecosystem. Many ecosystems prefer SemVer, and SemVer is generally preferred for application programmer interfaces (APIs) and software development kits (SDKs). CalVer tends to be used for projects that are large, have an unusually large number of independently-developed dependencies, have a constantly-changing scope, or are time-sensitive. It is SUGGESTED that those who use CalVer include a micro level value, because including a micro level supports simultaneously-maintained branches when that is necessary. Other version numbering formats MAY be used as version numbers, including git commit IDs or mercurial changeset IDs, as long as they uniquely identify versions. However, some alternatives (such as git commit IDs) can cause problems as release identifiers, because users may not be able to easily determine if they are up-to-date. The version ID format may be unimportant for identifying software releases if all recipients only run the latest version (e.g., it is the code for a single website or internet service that is constantly updated via continuous delivery).

Thoughts?

+1 - Works for me.


Popup text box not fixed under Light/Dark mode button.

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Steps to reproduce 🕹
Link to live example:
Steps: open the site by clicking https://deploy-preview-39077--material-ui.netlify.app/

Scroll down to the bottom in Sponsors area
2.Toggle the mode Light/Dark from the navbar
Scroll the page slight up and down then you will and unusual behaviour of the popup text box

https://github.com/mui/material-ui/assets/121485018/1e5c7af9-e910-4a69-b080-db1594a551e9
Current behavior 😯
The text box is floating up down unexpectedly
Expected behavior 🤔
The text box should not be floating with scrolling.
Context 🔦
This is a minor UI related issue but it is causing a slightly down the user impression while browsing the web page.
Your environment 🌎

npx @mui/envinfo
   System:
    OS: Windows 10 10.0.22621
  Binaries:
    Node: 18.16.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 9.5.1 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Chrome: Not Found
    Edge: Chromium (117.0.2045.43)

Can anyone help me to find the "Theme Mode Popup Component" in the code base cause I want to fix the UI bug

hey i want to fix this issue can you assign it to me

@NilanjanPramanik For issues related to PRs that have not been merged yet, I suggest you comment or report the issue in the PR directly
I will close this one for now!


Cannot import video files, only images

The "Import Data..." file chooser only allows selecting image files, but video files are not selectable.

It works for me. Can you confirm the same for you, too?

It appears to be fixed now.


How disable gem mruby-io

Is possible to turn off all disk function on mruby?
Acessing file is sometimes dangerous.
is possible add this in to turn off if needed?
MRuby::GemBox.new do |conf|
end

and socket

Reminds me a bit of my proposals in regards to easy static binaries of MRI ruby. I am glad that other people also have somewhat similar ideas. \o/
Like a containerized mini-ruby-OS.

@letarg0 Paste your build_config.rb or build log first.

@take-cheeze my build_config.rb is default. I'm no have option to disable mruby-io because it is not exist in build_config.rb .

https://github.com/mruby/mruby/blob/097bfdcac573861567716e4247265becec6b249a/mrbgems/default.gembox#L3
is the cause so remove all
  conf.gembox 'default'

and add mrbgems you use instead.

Define MRB_DISABLE_STDIO preprocessor macro somewhere too.

My conf is good?
I cant use File.open
https://gist.github.com/texrg/20e3ce008a653c26ef0b6e4610b22441

I'm not found mruby-io on default.gembox
This is good idea to remove all

Thanks

May I close the issue, or is there any remaining issue?

Yes


Docker Provider in swarm mode: Infer Traefik configuration from Service labels

Do you want to request a feature or report a bug?
Feature
What did you expect to see?

I think the best way to explain the feature I'm requesting is through a concrete example.
I would like to add something like these labels to a service (in this case a compose file):
labels:
  - traefik.http.routers.my-router.rule=...
  - traefik.http.middlewares.my-middleware.stripprefix.prefixes=...
  - traefik.http.services.my-service.loadbalancer.server.port=...

Then, when Traefik (in swarm mode) reads the labels off the service, it could generate the Traefik configuration that would be the equivalent of this:
labels:
  - traefik.http.routers.<swarm-service-name>-my-router.rule=...
  - traefik.http.routers.<swarm-service-name>-my-router.middlewares="<swarm-service-name>-my-middleware"
  - traefik.http.middlewares.<swarm-service-name>-my-middleware.stripprefix.prefixes=...
  - traefik.http.routers.<swarm-service-name>-my-router.service="<swarm-service-name>-my-service"
  - traefik.http.services.<swarm-service-name>-my-service.loadbalancer.server.port=...

In the context of the deploying a service, Traefik has enough information that it could simplify the configuration that needs to be added manually.
Doing that would allow deploying multiple stacks from the same compose file (see https://community.containo.us/t/traefik-v2-unique-deployment-labels-from-the-same-compose-file) and my guess is that it would be more intuitive to people (see https://community.containo.us/t/traefik-docker-label-scope/853/1).
This would also prevent the case where two teams that deploy into the same swarm accidentally choose the same router name and take down the other team's application.
The same thing could probably be done for the docker provider when not in swarm mode (using the container name), too.
Thanks!

标签:

traefik.http.routers.-my-router.rule=.
traefik.http.routers.-my-router.middlewares=“-my-middleware”
traefik.http.middlewares.-my-middleware.stripprefix.prefixes=.
traefik.http.routers.-my-router.service=“-my-service”
traefik.http.services.-my-service.loadbalancer.server.port=.

I'm also interested in this. For Swarm-based deployments, I originally thought I could get this done using service templates. But, it turns out that Swarm only supports templating hostname, mount, and environment properties, not labels.

Perhaps use * or some other single character to represent the current service name.  It would be much easier to catch and type.

I suspect this issue could also be interpreted as a bug, because apparently Traefik ignores services deployed in different stacks if they share the same service name.  For example, if I deploy two stacks to a swarm (stack foo and stack bar) which include services that share the same name (for example, frontend)  then, although Docker lists two independent services running well (i.e., foo_frontend and bar_frontend), traefik starts to behave poorly and ultimately cuts routes to the services launched as part of those stacks.

Before traefik:v2 I just have to add this deploy labels and execute docker stack deploy multiple times :
etherpad:
  image: etherpad/etherpad:latest
  deploy:
    labels:
      traefik.port: 80
      traefik.frontend.rule: Host:${DOMAIN:?err}
      traefik.enable: "true"

Now I can't deploy multiple etherpad instances :(
etherpad:
  image: etherpad/etherpad:latest
  labels:
    - traefik.enable=true
    - traefik.http.routers.etherpad.rule=Host(`${DOMAIN:-etherpad.localhost}`)
    - traefik.http.routers.etherpad.tls=${HTTPS:-false}
    - traefik.http.routers.etherpad.tls.certresolver=mytlschallenge

I have an error : Router defined multiple times with different configurations

I have an error : Router defined multiple times with different configurations

We tried working around this by expanding the default rule setting at the provider level. That works with regard to rules, but does not allow any other settings for the generated router to be changed.
Example:
--providers.docker.defaultrule='{{ index .Labels "whatever.you.want.to.hold.your.container.rule" }}'

But obviously this requires you to have that label defined on every service, and still does not allow changing e.g. TLS or entrypoints settings per service.

Turns out these lines make it work well with Docker Compose, but fail to make it work with Swarm:
https://github.com/containous/traefik/blob/cac76a182ef68e4f9739632337fafa18fc9e17df/pkg/provider/docker/config.go#L327-L335
If a few lines were added to allow setting the service name from the com.docker.swarm.service.name label, the Traefik services for the different containers would be merged, resulting in a single Traefik service per Swarm service, and thus enabling the full configuration of the router for the Swarm service using labels. It would also alleviate the Router defined multiple times with different configurations error.
CC @ston3o @kinghuang.

I wonder... is it possible to use Go templating in docker container labels?: https://docs.traefik.io/providers/file/#go-templating

I wonder... is it possible to use Go templating in docker container labels?: https://docs.traefik.io/providers/file/#go-templating

It is possible to use it for the label values, but not their names. And the template data available is fairly limited:

Placeholder
Description

.Service.ID
Service ID

.Service.Name
Service name

.Service.Labels
Service labels

.Node.ID
Node ID

.Node.Hostname
Node Hostname

.Task.ID
Task ID

.Task.Name
Task name

.Task.Slot
Task slot

https://docs.docker.com/engine/reference/commandline/service_create/#create-services-using-templates

I guess if we'd be able to use go templating in label keys and we could use things like the container name, that'd fix the problem.

@Yajo The only way to achieve that today is using docker-compose and by using the list form for entering labels, which allows for substituting environment variables at config/deploy time. There is no runtime templating option though. I've managed to suffice with substituting in e.g. the service and environment names in the label keys this way, and setting rules and server ports this way. Though it would be great not having to come up with banking schemes when the names likely could be inferred from the remaining context, e.g. service name labels such as those provided by docker-compose or docker stack deploy.

PS: FWIW, the "Task Name" in the table above is in practice identical to the container name. An alternative to having Docker implement templating in container label keys would be for Traefik to support a placeholder or something like that. I have tried with the Go templating that does take place, but out requires spaces in the template expression to look up labels, and the label keys cannot contain spaces by design.

I'd love to have a wildcard for the current service name aswell...we configure many services in docker-compose and it happened multiple times that developers copy and pasted labels and forgot to change the label name resulting in unexpected behaviour. It's one of the reasons we have not completly migrated to v2 yet.
Something like that would really be nice:
traefik.http.routers.*.rule

Really the simpliest way is to just treat the traefik.http.routers.my-router.rule values as go templates like is already happening with https://docs.traefik.io/v2.2/providers/docker/#defaultrule
I thought that would have been logical?

nvm i realised the key needs to be templatable too or it just overwrites the previous deploy

This feature is the only reason why I stay on V1...
Any news ?

###The below method works best for me, I have exported the env variables before the stack deploy using Jenkins pipeline
and then mentioned in the docker-compose file.
it works only for this format only ( - "traefik.XXXXX=XXX")
version: "3.7"
services:
whoami:
image: containous/whoami:v1.3.0
networks:
- traefik-public
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.${ROUTER}.rule=Host($DOMAIN.example.com)"
- "traefik.http.routers.${ROUTER}.middlewares=authtraefik"
- "traefik.http.routers.${ROUTER}.entrypoints=https"
- "traefik.http.routers.${ROUTER}.tls=true"
- "traefik.http.routers.${ROUTER}.tls.certresolver=myresolver"
# Basic Auth
- "traefik.http.middlewares.authtraefik.basicauth.users=user:$$apr1$$q8eZFHjF$$Fvmkk//V6Btlaf2i/ju5n/"
# Swarm Mode
- "traefik.http.services.${ROUTER}.loadbalancer.server.port=80"
networks:
traefik-public:
external: true

@vicknesh22,
Hurray! It works! That's exactly what I was looking for. I still think it would be great and intuitive if the labels defined on a service weren't global, so I'll keep the GitHub issue I created open, but this solves the most annoying (maybe too strong of a word) part of integrating with our dynamic deployments.
The entire time since I opened the issue issue I've been deploying stacks at a dynamic location by just leaving the labels off the compose file and adding them as an additional deployment step after the docker stack deploy... by running docker service update....  Having them back in the compose file is much cleaner now.
Thanks for taking the time to post how to do it!

This is missing in the traefik document.
Hope they comesup with a permanent fix soon!!!

+1 for adding token replacement of container ID functionality to the labels.
Currently everyone has to fall back on environment variables in the labels to pass them a unique ID. For my use case, to deploy development environments I need to use that unique ID in the Host rule also.
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.$UNIQUE_ID-ide.rule=Host(`$UNIQUE_ID.docker.localhost`) && PathPrefix(`/ide`)"
      - "traefik.http.routers.$UNIQUE_ID-ide.service=$UNIQUE_ID-ide"
      - "traefik.http.routers.$UNIQUE_ID-ide.middlewares=ide-stripprefix"
      - "traefik.http.middlewares.ide-stripprefix.stripprefix.prefixes=/ide"
      - "traefik.http.services.$UNIQUE_ID-ide.loadbalancer.server.port=3000"
      - "traefik.http.routers.$UNIQUE_ID-web.rule=Host(`$UNIQUE_ID.docker.localhost`)"
      - "traefik.http.routers.$UNIQUE_ID-web.service=$UNIQUE_ID-web"
      - "traefik.http.services.$UNIQUE_ID-web.loadbalancer.server.port=80"

It would be super nice if we didn't have to generate that unique ID before actually starting the container. Docker gives all containers a unique ID, so a token that's able to reference that ID looks ideal to me.

I have the same issue that I've mentioned in #7395 , here's the comment:
Currently, AWS allows only for specifying Docker labels when creating a task definition i.e. you can't modify those when running a task. Which is a problem in my usecase where I have a single task definition which I start for every user. To make it simple, imagine a Wordpress image which is the same for everyone and just differs by the hostname. However, when I start the task (not a service), I give it appropriate tags to differentiate from the other tasks.
Which is a general problem I believe when running the same task definition which then differs only on tags/env/domain variables. For example, I have no way to set service parameters such as port as I have to put that in the dockerLabels what means that I can't use dynamic service name anymore

I have the same issue that I've mentioned in #7395 , here's the comment:
Currently, AWS allows only for specifying Docker labels when creating a task definition i.e. you can't modify those when running a task. Which is a problem in my usecase where I have a single task definition which I start for every user. To make it simple, imagine a Wordpress image which is the same for everyone and just differs by the hostname. However, when I start the task (not a service), I give it appropriate tags to differentiate from the other tasks.
Which is a general problem I believe when running the same task definition which then differs only on tags/env/domain variables. For example, I have no way to set service parameters such as port as I have to put that in the dockerLabels what means that I can't use dynamic service name anymore

The solution above breaks down when you have more than one service using Traefik defined in compose / swarm.
Really, for my purposes, my routers / http.services will never be named anything other than ${COMPOSE_PROJECT_NAME}-${COMPOSE-SERVICE-NAME}, both of which are already applied as labels to my containers via Docker Compose. It would be great if Traefik could infer these things.


[Bug 1208368] Force remove to all group members on invalidation day.

@akatsoulas r?

Coverage decreased (-0.01%) to 92.424% when pulling 80209e213f666e7281dac128c12878ffff87f56e on johngian:fix-invalidation into cc6035640cd813588f854f3cf7e5cbc65ba963f9 on mozilla:master.

r+wc

Changes Unknown when pulling e2e388142a58029b616771ddc59570624cb724a6 on johngian:fix-invalidation into ** on mozilla:master**.


Add libtorch in the CMake

Dear all,
I am doing libtorch for LSTM with one of the demo, how can I modify the CMakeList.txt?
I tried the demos/CMakeList.txt but I cannot manage to compile.
I hope someone could suggest me the method.
Thanks a lot

I've never tried libtorch, but according to https://pytorch.org/cppdocs/installing.html#minimal-example it looks that the simple (ugly) solution can be
add
find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

at https://github.com/opencv/open_model_zoo/blob/f9c2ed239724506c855ab0cb91ba84d12040206e/demos/CMakeLists.txt#L210
replace set (CMAKE_CXX_STANDARD 11) with set (CMAKE_CXX_STANDARD 14)
delete if (${CMAKE_CXX_COMPILER_ID} STREQUAL GNU) and its body,
and add target_link_libraries(${IE_SAMPLE_NAME} "${TORCH_LIBRARIES}") to https://github.com/opencv/open_model_zoo/blob/f9c2ed239724506c855ab0cb91ba84d12040206e/demos/CMakeLists.txt#L186
You can add that if (MSVC) from libtorch example after target_link_libraries(${IE_SAMPLE_NAME} "${TORCH_LIBRARIES}").
This should make libtorch available to every demo.
But that's impossible to say where you have stuck without you describing what you did and providing an error message.
It may also be possible to convert the model into Intermediate Representation used by OpenVINO's Inference Engine, so you won't need integrating libtorch.

Thanks for the help
when i add this line target_link_libraries(${IE_SAMPLE_NAME} "${TORCH_LIBRARIES}"),
the error is the keyword signature for target_link_libraries has been used with...
Then I change that line to  target_link_libraries(${IE_SAMPLE_NAME} PRIVATE ${TORCH_LIBRARIES}), and it could be compiled, but the demo still cannot work with libtorch.
I believe that I need to set the CMAKE_PREFIX_PATH but where can I insert it? since I am just using the build_demos.sh, I have change that cmake to cmake -DCMAKE_PREFIX_PATH=~/libtorch -DCMAKE_BUILD_TYPE=Release but it does not work
Regarding the pytorch page, we need to add the prefix at cmake, do you have any idea?

CMAKE_PREFIX_PATH helps to find_package(Torch REQUIRED). But if cmake passes for you without setting CMAKE_PREFIX_PATH, than cmake is able to find libtorch even without it. But yes, to set CMAKE_PREFIX_PATH in build_demos.sh you need to replace (cd "$build_dir" && cmake -DCMAKE_BUILD_TYPE=Release "${extra_cmake_opts[@]}" "$DEMOS_PATH") with (cd "$build_dir" && cmake -DCMAKE_PREFIX_PATH=~/libtorch -DCMAKE_BUILD_TYPE=Release "${extra_cmake_opts[@]}" "$DEMOS_PATH")
Again, you can't get a lot of help, unless you specify how you know that "the demo still cannot work with libtorch" and "cmake -DCMAKE_PREFIX_PATH=~/libtorch -DCMAKE_BUILD_TYPE=Release but it does not work".

Thanks for the solution, I have tried out with torch tensor to fit in the model.
the error is coming from the model saving method from python, not the CMake issue.
Thanks again your help!


disable observers without cross-thread side effects

The current implementation of enabling/disabling observers has side effects into other threads.
I understand that the set of observers per model should be global, but disabling an observer is usually done for a specific reason and a specific scope.
When doing this:
    Heisenberg.observers.disable :uncle_frank do
        #code with observer disabled
    end

Other threads using the Heisenberg model will have the observer disabled until the block is done with it.
This pull requests doesn't change the global behaviour of observers, only the disabling/enabling is per thread.

:+1: this is a problem for us in production and our current solution is to fork whenever we need to disable callbacks. :(
If you still would like to see changes to this and @markmeeus isn't available or doesn't have time to make them I would be glad to take over the PR.

As an update, we have been running this code in production for many many months now with no problems.

awesome!


[CON-3775] Change translations

I think you have to rename config/import/overwrite_properties_helptext to something else, because if the plugin is already installed this snippet won't be changed. It's not released yet, but QA can fail because of that.


Crash when changing a drive

Describe the bug
Just installed the app and tried to switch drives. App became very laggy and, after switching, it crashed.
Didn't happen again after, but should still be fixed.
To Reproduce
Steps to reproduce the behavior:

Click on any drive
Sometimes the app crashes

Expected behavior
The app doesn't crash
Desktop (please complete the following information):

OS: Windows 10 x64 1909 18363.836
App version: v0.9.0 Beta

Additional context
Log:
2020-05-27 22:32:40.6527|INFO|Files.Filesystem.DrivesManager|Drive added: C:\, Fixed
2020-05-27 22:32:40.6827|INFO|Files.Filesystem.DrivesManager|Drive added: D:\, Fixed
2020-05-27 22:32:40.7137|INFO|Files.Filesystem.DrivesManager|Drive added: E:\, Fixed
2020-05-27 22:32:40.7437|INFO|Files.Filesystem.DrivesManager|Drive added: F:\, Fixed
2020-05-27 22:37:40.3122|ERROR|Files.App|A method was called at an unexpected time.

May or may not be related to #871. It crashed again right after switching from that directory with the same error in the log.

Thank you for creating this feedback item. This issue is currently resolved in our active development branch and will be shipped soon once we've validated that everything works properly. Please subscribe to our releases to be notified when this is available.
If you're still seeing this one after re-testing with the necessary update, we'd be thrilled if you could reopen this issue.


Nest an Anchor Element within a Paragraph

Challenge Waypoint: Nest an Anchor Element within a Paragraph has an issue.
User Agent is: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:
<link href="http://fonts.googleapis.com/css?family=Lobster" rel="stylesheet" type="text/css">

CatPhotoApp

cat photos

Kitty ipsum dolor sit amet, shed everywhere shed everywhere stretching attack your ankles chase the red dot, hairball run catnip eat the grass sniff.

Purr jump eat the grass rip the couch scratched sunbathe, shed everywhere rip the couch sleep in the sink fluffy fur catnip scratched.

View more cat photos

Rather than creating a new anchor tag wrapped in a paragraph tag at the bottom of the page, the challenge is asking you to wrap the one already on the page (above the img tag) in a paragraph. Because the existing anchor tag comes before either of the provided paragraphs the test needs your new paragraph to be the first one on the page.
Closing this as not an issue with the tests, but please feel free to reply if you need further clarification.
Thanks and happy coding!


Interface generator tool design doc

Description

Design document for an interface auto generator script. This removes the need to manually create and modify interfaces after the creation/modification of a contract.
Additional context

More context: https://github.com/ethereum-optimism/optimism/issues/12412
Metadata

Fixes https://github.com/ethereum-optimism/optimism/issues/12413

It might be good to link this doc in here somewhere: https://github.com/ethereum-optimism/optimism/blob/develop/packages/contracts-bedrock/INTERFACES.md


Does sarama support renaming consumer group?

Question.
Kafka supports renaming consumer-group as mentioned here
I have a use case in which we need to rename a consumer group in Kafka. Please suggest if it is possible using sarama?

@alok87 have you found the solution?


build-using-dockerfile: add --annotation

Add an --annotation flag to "buildah build-using-dockerfile".
Refactor the logic for --label handling to use SetLabel() to set them in the image configuration in Executor.Commit(), instead of passing them as a field in CommitOptions for Builder.Commit() and expecting it to do so.

LGTM

LGTM, F27 rpm test in Travis is misbehaving, going to retest.

bot, retest this please

@rh-atomic-bot r+