[haiku-commits] r35393 - in haiku/trunk: headers/private/kernel/vm src/add-ons/kernel/bus_managers/agp_gart src/system/kernel src/system/kernel/arch/ppc src/system/kernel/arch/x86 ...
- From: ingo_weinhold@xxxxxx
- To: haiku-commits@xxxxxxxxxxxxx
- Date: Wed, 3 Feb 2010 19:53:53 +0100 (CET)
Author: bonefish
Date: 2010-02-03 19:53:52 +0100 (Wed, 03 Feb 2010)
New Revision: 35393
Changeset: http://dev.haiku-os.org/changeset/35393/haiku
Ticket: http://dev.haiku-os.org/ticket/5328
Removed:
haiku/trunk/src/system/kernel/vm/vm_daemons.cpp
Modified:
haiku/trunk/headers/private/kernel/vm/VMTranslationMap.h
haiku/trunk/headers/private/kernel/vm/vm.h
haiku/trunk/headers/private/kernel/vm/vm_page.h
haiku/trunk/headers/private/kernel/vm/vm_priv.h
haiku/trunk/headers/private/kernel/vm/vm_types.h
haiku/trunk/src/add-ons/kernel/bus_managers/agp_gart/agp_gart.cpp
haiku/trunk/src/system/kernel/arch/ppc/arch_vm_translation_map.cpp
haiku/trunk/src/system/kernel/arch/x86/X86VMTranslationMap.h
haiku/trunk/src/system/kernel/arch/x86/arch_vm_translation_map.cpp
haiku/trunk/src/system/kernel/cache/file_cache.cpp
haiku/trunk/src/system/kernel/low_resource_manager.cpp
haiku/trunk/src/system/kernel/slab/MemoryManager.cpp
haiku/trunk/src/system/kernel/vm/Jamfile
haiku/trunk/src/system/kernel/vm/PageCacheLocker.h
haiku/trunk/src/system/kernel/vm/VMCache.cpp
haiku/trunk/src/system/kernel/vm/VMTranslationMap.cpp
haiku/trunk/src/system/kernel/vm/vm.cpp
haiku/trunk/src/system/kernel/vm/vm_page.cpp
Log:
* Removed useless return parameter from vm_remove_all_page_mappings().
* Added vm_clear_page_mapping_accessed_flags() and
vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality
of vm_test_map_activation(), vm_clear_map_flags(), and
vm_remove_all_page_mappings(), thus saving lots of calls to translation map
methods. The backend is the new method
VMTranslationMap::ClearAccessedAndModified().
* Started to make use of the cached page queue and changed the meaning of the
other non-free queues slightly:
- Active queue: Contains mapped pages that have been used recently.
- Inactive queue: Contains mapped pages that have not been used recently. Also
contains unmapped temporary pages.
- Modified queue: Contains unmapped modified pages.
- Cached queue: Contains unmapped unmodified pages (LRU sorted).
Unless we're actually low on memory and actively do paging, modified and
cached queues only contain non-temporary pages. Cached pages are considered
quasi free. They still belong to a cache, but since they are unmodified and
unmapped, they can be freed immediately. And this is what
vm_page_[try_]reserve_pages() do now when there are no more actually free
pages at hand. Essentially this means that pages storing cached file data,
unless mmap()ped, no longer are considered used and don't contribute to page
pressure. Paging will not happen as long there are enough free + cached pages
available.
* Reimplemented the page daemon. It no longer scans all pages, but instead works
the page queues. As long as the free pages situation is harmless, it only
iterates through the active queue and deactivates pages that have not been
used recently. When paging occurs it additionally scans the inactive queue and
frees pages that have not been used recently.
* Changed the page reservation/allocation interface:
vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and
vm_page_allocate_page() now take a vm_page_reservation structure pointer.
The reservation functions initialize the structure -- currently consisting
only of a count member for the number of still reserved pages.
vm_page_allocate_page() decrements the count and vm_page_unreserve_pages()
unreserves the remaining pages (if any). Advantages are that reservation/
unreservation mismatches cannot occur anymore, that vm_page_allocate_page()
can verify that the caller has indeed a reserved page left, and that there's
no unnecessary pressure on the free page pool anymore. The only disadvantage
is that the vm_page_reservation object needs to be passed around a bit.
* Reworked the page reservation implementation:
- Got rid of sSystemReservedPages and sPageDeficit. Instead
sUnreservedFreePages now actually contains the number of free pages that
have not yet been reserved (it cannot become negative anymore) and the new
sUnsatisfiedPageReservations contains the number of pages that are still
needed for reservation.
- Threads waiting for reservations do now add themselves to a waiter queue,
which is ordered by descending priority (VM priority and thread priority).
High priority waiters are served first when pages become available.
Fixes #5328.
* cache_prefetch_vnode(): Would reserve one less page than allocated later, if
the size wasn't page aligned.
Modified: haiku/trunk/headers/private/kernel/vm/VMTranslationMap.h
===================================================================
--- haiku/trunk/headers/private/kernel/vm/VMTranslationMap.h 2010-02-03
18:47:07 UTC (rev 35392)
+++ haiku/trunk/headers/private/kernel/vm/VMTranslationMap.h 2010-02-03
18:53:52 UTC (rev 35393)
@@ -14,6 +14,7 @@
struct kernel_args;
+struct vm_page_reservation;
struct VMArea;
@@ -32,13 +33,15 @@
virtual status_t Map(addr_t virtualAddress,
addr_t
physicalAddress,
- uint32
attributes) = 0;
+ uint32
attributes,
+
vm_page_reservation* reservation) = 0;
virtual status_t Unmap(addr_t start, addr_t end)
= 0;
// map not locked
- virtual status_t UnmapPage(VMArea* area, addr_t
address) = 0;
+ virtual status_t UnmapPage(VMArea* area, addr_t
address,
+ bool
updatePageQueue) = 0;
virtual void UnmapPages(VMArea* area, addr_t
base,
- size_t
size);
+ size_t
size, bool updatePageQueue);
virtual void UnmapArea(VMArea* area,
bool
deletingAddressSpace,
bool
ignoreTopCachePageFlags);
@@ -60,6 +63,11 @@
virtual status_t ClearFlags(addr_t
virtualAddress,
uint32
flags) = 0;
+ virtual bool ClearAccessedAndModified(
+ VMArea*
area, addr_t address,
+ bool
unmapIfUnaccessed,
+ bool&
_modified) = 0;
+
virtual void Flush() = 0;
protected:
Modified: haiku/trunk/headers/private/kernel/vm/vm.h
===================================================================
--- haiku/trunk/headers/private/kernel/vm/vm.h 2010-02-03 18:47:07 UTC (rev
35392)
+++ haiku/trunk/headers/private/kernel/vm/vm.h 2010-02-03 18:53:52 UTC (rev
35393)
@@ -109,9 +109,10 @@
status_t vm_set_area_memory_type(area_id id, addr_t physicalBase, uint32 type);
status_t vm_get_page_mapping(team_id team, addr_t vaddr, addr_t *paddr);
bool vm_test_map_modification(struct vm_page *page);
-int32 vm_test_map_activation(struct vm_page *page, bool *_modified);
void vm_clear_map_flags(struct vm_page *page, uint32 flags);
-void vm_remove_all_page_mappings(struct vm_page *page, uint32 *_flags);
+void vm_remove_all_page_mappings(struct vm_page *page);
+int32 vm_clear_page_mapping_accessed_flags(struct vm_page *page);
+int32 vm_remove_all_page_mappings_if_unaccessed(struct vm_page *page);
status_t vm_get_physical_page(addr_t paddr, addr_t* vaddr, void** _handle);
status_t vm_put_physical_page(addr_t vaddr, void* handle);
Modified: haiku/trunk/headers/private/kernel/vm/vm_page.h
===================================================================
--- haiku/trunk/headers/private/kernel/vm/vm_page.h 2010-02-03 18:47:07 UTC
(rev 35392)
+++ haiku/trunk/headers/private/kernel/vm/vm_page.h 2010-02-03 18:53:52 UTC
(rev 35393)
@@ -16,6 +16,12 @@
extern int32 gMappedPagesCount;
+
+struct vm_page_reservation {
+ uint32 count;
+};
+
+
#ifdef __cplusplus
extern "C" {
#endif
@@ -45,11 +51,14 @@
void vm_page_schedule_write_page_range(struct VMCache *cache,
uint32 firstPage, uint32 endPage);
-void vm_page_unreserve_pages(uint32 count);
-void vm_page_reserve_pages(uint32 count, int priority);
-bool vm_page_try_reserve_pages(uint32 count, int priority);
+void vm_page_unreserve_pages(vm_page_reservation* reservation);
+void vm_page_reserve_pages(vm_page_reservation* reservation, uint32 count,
+ int priority);
+bool vm_page_try_reserve_pages(vm_page_reservation* reservation, uint32 count,
+ int priority);
-struct vm_page *vm_page_allocate_page(uint32 flags);
+struct vm_page *vm_page_allocate_page(vm_page_reservation* reservation,
+ uint32 flags);
struct vm_page *vm_page_allocate_page_run(uint32 flags, addr_t base,
addr_t length, int priority);
struct vm_page *vm_page_allocate_page_run_no_base(uint32 flags, addr_t count,
Modified: haiku/trunk/headers/private/kernel/vm/vm_priv.h
===================================================================
--- haiku/trunk/headers/private/kernel/vm/vm_priv.h 2010-02-03 18:47:07 UTC
(rev 35392)
+++ haiku/trunk/headers/private/kernel/vm/vm_priv.h 2010-02-03 18:53:52 UTC
(rev 35393)
@@ -31,7 +31,6 @@
bool isUser, addr_t *newip);
void vm_unreserve_memory(size_t bytes);
status_t vm_try_reserve_memory(size_t bytes, int priority, bigtime_t timeout);
-void vm_schedule_page_scanner(uint32 target);
status_t vm_daemon_init(void);
const char *page_state_to_string(int state);
Modified: haiku/trunk/headers/private/kernel/vm/vm_types.h
===================================================================
--- haiku/trunk/headers/private/kernel/vm/vm_types.h 2010-02-03 18:47:07 UTC
(rev 35392)
+++ haiku/trunk/headers/private/kernel/vm/vm_types.h 2010-02-03 18:53:52 UTC
(rev 35393)
@@ -113,7 +113,7 @@
bool modified : 1;
uint8 unused : 1;
- int8 usage_count;
+ uint8 usage_count;
uint16 wired_count;
VMCacheRef* CacheRef() const { return cache_ref; }
Modified: haiku/trunk/src/add-ons/kernel/bus_managers/agp_gart/agp_gart.cpp
===================================================================
--- haiku/trunk/src/add-ons/kernel/bus_managers/agp_gart/agp_gart.cpp
2010-02-03 18:47:07 UTC (rev 35392)
+++ haiku/trunk/src/add-ons/kernel/bus_managers/agp_gart/agp_gart.cpp
2010-02-03 18:53:52 UTC (rev 35393)
@@ -550,12 +550,13 @@
if (memory->pages == NULL)
return B_NO_MEMORY;
- vm_page_reserve_pages(count, VM_PRIORITY_SYSTEM);
+ vm_page_reservation reservation;
+ vm_page_reserve_pages(&reservation, count, VM_PRIORITY_SYSTEM);
for (uint32 i = 0; i < count; i++) {
- memory->pages[i] = vm_page_allocate_page(
+ memory->pages[i] = vm_page_allocate_page(&reservation,
PAGE_STATE_WIRED | VM_PAGE_ALLOC_CLEAR);
}
- vm_page_unreserve_pages(count);
+ vm_page_unreserve_pages(&reservation);
}
#ifdef DEBUG_PAGE_ACCESS
Modified: haiku/trunk/src/system/kernel/arch/ppc/arch_vm_translation_map.cpp
===================================================================
--- haiku/trunk/src/system/kernel/arch/ppc/arch_vm_translation_map.cpp
2010-02-03 18:47:07 UTC (rev 35392)
+++ haiku/trunk/src/system/kernel/arch/ppc/arch_vm_translation_map.cpp
2010-02-03 18:53:52 UTC (rev 35393)
@@ -147,10 +147,12 @@
virtual status_t Map(addr_t virtualAddress,
addr_t
physicalAddress,
- uint32
attributes);
+ uint32
attributes,
+
vm_page_reservation* reservation);
virtual status_t Unmap(addr_t start, addr_t end);
- virtual status_t UnmapPage(VMArea* area, addr_t
address);
+ virtual status_t UnmapPage(VMArea* area, addr_t
address,
+ bool
updatePageQueue);
virtual status_t Query(addr_t virtualAddress,
addr_t*
_physicalAddress,
@@ -164,6 +166,11 @@
virtual status_t ClearFlags(addr_t
virtualAddress,
uint32
flags);
+ virtual bool ClearAccessedAndModified(
+ VMArea*
area, addr_t address,
+ bool
unmapIfUnaccessed,
+ bool&
_modified);
+
virtual void Flush();
protected:
@@ -389,7 +396,7 @@
status_t
PPCVMTranslationMap::Map(addr_t virtualAddress, addr_t physicalAddress,
- uint32 attributes)
+ uint32 attributes, vm_page_reservation* reservation)
{
// lookup the vsid based off the va
uint32 virtualSegmentID = VADDR_TO_VSID(fVSIDBase, virtualAddress);
@@ -463,7 +470,8 @@
status_t
-PPCVMTranslationMap::UnmapPage(VMArea* area, addr_t address)
+PPCVMTranslationMap::UnmapPage(VMArea* area, addr_t address,
+ bool updatePageQueue)
{
ASSERT(address % B_PAGE_SIZE == 0);
@@ -513,10 +521,20 @@
} else
page->wired_count--;
- if (page->wired_count == 0 && page->mappings.IsEmpty())
+ locker.Unlock();
+
+ if (page->wired_count == 0 && page->mappings.IsEmpty()) {
atomic_add(&gMappedPagesCount, -1);
- locker.Unlock();
+ if (updatePageQueue) {
+ if (page->Cache()->temporary)
+ vm_page_set_state(page, PAGE_STATE_INACTIVE);
+ else if (page->modified)
+ vm_page_set_state(page, PAGE_STATE_MODIFIED);
+ else
+ vm_page_set_state(page, PAGE_STATE_CACHED);
+ }
+ }
if (mapping != NULL) {
bool isKernelSpace = area->address_space ==
VMAddressSpace::Kernel();
@@ -604,6 +622,52 @@
}
+bool
+PPCVMTranslationMap::ClearAccessedAndModified(VMArea* area, addr_t address,
+ bool unmapIfUnaccessed, bool& _modified)
+{
+ // TODO: Implement for real! ATM this is just an approximation using
+ // Query(), ClearFlags(), and UnmapPage(). See below!
+
+ RecursiveLocker locker(fLock);
+
+ uint32 flags;
+ addr_t physicalAddress;
+ if (Query(address, &physicalAddress, &flags) != B_OK
+ || (flags & PAGE_PRESENT) == 0) {
+ return false;
+ }
+
+ _modified = (flags & PAGE_MODIFIED) != 0;
+
+ if ((flags & (PAGE_ACCESSED | PAGE_MODIFIED)) != 0)
+ ClearFlags(address, flags & (PAGE_ACCESSED | PAGE_MODIFIED));
+
+ if ((flags & PAGE_ACCESSED) != 0)
+ return true;
+
+ if (!unmapIfUnaccessed)
+ return false;
+
+ locker.Unlock();
+
+ UnmapPage(area, address, false);
+ // TODO: Obvious race condition: Between querying and unmapping
the
+ // page could have been accessed. We try to compensate by
considering
+ // vm_page::{accessed,modified} (which would have been updated
by
+ // UnmapPage()) below, but that doesn't quite match the required
+ // semantics of the method.
+
+ vm_page* page = vm_lookup_page(physicalAddress / B_PAGE_SIZE);
+ if (page == NULL)
+ return false;
+
+ _modified |= page->modified;
+
+ return page->accessed;
+}
+
+
void
PPCVMTranslationMap::Flush()
{
@@ -786,11 +850,15 @@
PPCVMTranslationMap* map = static_cast<PPCVMTranslationMap*>(
addressSpace->TranslationMap());
+ vm_page_reservation reservation;
+ vm_page_reserve_pages(&reservation, 0, VM_PRIORITY_USER);
+ // We don't need any pages for mapping.
+
// map the pages
for (; virtualAddress < virtualEnd;
virtualAddress += B_PAGE_SIZE, physicalAddress += B_PAGE_SIZE)
{
status_t error = map->Map(virtualAddress, physicalAddress,
- B_KERNEL_READ_AREA | B_KERNEL_WRITE_AREA);
+ B_KERNEL_READ_AREA | B_KERNEL_WRITE_AREA, &reservation);
if (error != B_OK)
return error;
}
Modified: haiku/trunk/src/system/kernel/arch/x86/X86VMTranslationMap.h
===================================================================
--- haiku/trunk/src/system/kernel/arch/x86/X86VMTranslationMap.h
2010-02-03 18:47:07 UTC (rev 35392)
+++ haiku/trunk/src/system/kernel/arch/x86/X86VMTranslationMap.h
2010-02-03 18:53:52 UTC (rev 35393)
@@ -31,12 +31,14 @@
virtual status_t Map(addr_t virtualAddress,
addr_t
physicalAddress,
- uint32
attributes);
+ uint32
attributes,
+
vm_page_reservation* reservation);
virtual status_t Unmap(addr_t start, addr_t end);
- virtual status_t UnmapPage(VMArea* area, addr_t
address);
+ virtual status_t UnmapPage(VMArea* area, addr_t
address,
+ bool
updatePageQueue);
virtual void UnmapPages(VMArea* area, addr_t
base,
- size_t
size);
+ size_t
size, bool updatePageQueue);
virtual void UnmapArea(VMArea* area,
bool
deletingAddressSpace,
bool
ignoreTopCachePageFlags);
@@ -54,6 +56,11 @@
virtual status_t ClearFlags(addr_t
virtualAddress,
uint32
flags);
+ virtual bool ClearAccessedAndModified(
+ VMArea*
area, addr_t address,
+ bool
unmapIfUnaccessed,
+ bool&
_modified);
+
virtual void Flush();
protected:
Modified: haiku/trunk/src/system/kernel/arch/x86/arch_vm_translation_map.cpp
===================================================================
--- haiku/trunk/src/system/kernel/arch/x86/arch_vm_translation_map.cpp
2010-02-03 18:47:07 UTC (rev 35392)
+++ haiku/trunk/src/system/kernel/arch/x86/arch_vm_translation_map.cpp
2010-02-03 18:53:52 UTC (rev 35393)
@@ -401,7 +401,8 @@
status_t
-X86VMTranslationMap::Map(addr_t va, addr_t pa, uint32 attributes)
+X86VMTranslationMap::Map(addr_t va, addr_t pa, uint32 attributes,
+ vm_page_reservation* reservation)
{
TRACE(("map_tmap: entry pa 0x%lx va 0x%lx\n", pa, va));
@@ -422,7 +423,8 @@
vm_page *page;
// we need to allocate a pgtable
- page = vm_page_allocate_page(PAGE_STATE_WIRED |
VM_PAGE_ALLOC_CLEAR);
+ page = vm_page_allocate_page(reservation,
+ PAGE_STATE_WIRED | VM_PAGE_ALLOC_CLEAR);
DEBUG_PAGE_ACCESS_END(page);
@@ -533,7 +535,8 @@
This object shouldn't be locked.
*/
status_t
-X86VMTranslationMap::UnmapPage(VMArea* area, addr_t address)
+X86VMTranslationMap::UnmapPage(VMArea* area, addr_t address,
+ bool updatePageQueue)
{
ASSERT(address % B_PAGE_SIZE == 0);
@@ -620,10 +623,20 @@
} else
page->wired_count--;
- if (page->wired_count == 0 && page->mappings.IsEmpty())
+ locker.Unlock();
+
+ if (page->wired_count == 0 && page->mappings.IsEmpty()) {
atomic_add(&gMappedPagesCount, -1);
- locker.Unlock();
+ if (updatePageQueue) {
+ if (page->Cache()->temporary)
+ vm_page_set_state(page, PAGE_STATE_INACTIVE);
+ else if (page->modified)
+ vm_page_set_state(page, PAGE_STATE_MODIFIED);
+ else
+ vm_page_set_state(page, PAGE_STATE_CACHED);
+ }
+ }
if (mapping != NULL) {
bool isKernelSpace = area->address_space ==
VMAddressSpace::Kernel();
@@ -637,7 +650,8 @@
void
-X86VMTranslationMap::UnmapPages(VMArea* area, addr_t base, size_t size)
+X86VMTranslationMap::UnmapPages(VMArea* area, addr_t base, size_t size,
+ bool updatePageQueue)
{
page_directory_entry* pd = fArchData->pgdir_virt;
@@ -720,13 +734,23 @@
} else
page->wired_count--;
- if (page->wired_count == 0 &&
page->mappings.IsEmpty())
+ if (page->wired_count == 0 &&
page->mappings.IsEmpty()) {
atomic_add(&gMappedPagesCount, -1);
+
+ if (updatePageQueue) {
+ if (page->Cache()->temporary)
+ vm_page_set_state(page,
PAGE_STATE_INACTIVE);
+ else if (page->modified)
+ vm_page_set_state(page,
PAGE_STATE_MODIFIED);
+ else
+ vm_page_set_state(page,
PAGE_STATE_CACHED);
+ }
+ }
}
}
Flush();
- // flush explicitely, since we directly use the lock
+ // flush explicitly, since we directly use the lock
pinner.Unlock();
}
@@ -752,7 +776,7 @@
bool ignoreTopCachePageFlags)
{
if (area->cache_type == CACHE_TYPE_DEVICE || area->wiring != B_NO_LOCK)
{
- X86VMTranslationMap::UnmapPages(area, area->Base(),
area->Size());
+ X86VMTranslationMap::UnmapPages(area, area->Base(),
area->Size(), true);
return;
}
@@ -770,10 +794,22 @@
vm_page* page = mapping->page;
page->mappings.Remove(mapping);
- if (page->wired_count == 0 && page->mappings.IsEmpty())
+ VMCache* cache = page->Cache();
+
+ if (page->wired_count == 0 && page->mappings.IsEmpty()) {
atomic_add(&gMappedPagesCount, -1);
- if (unmapPages || page->Cache() != area->cache) {
+ if (!ignoreTopCachePageFlags || cache != area->cache) {
+ if (cache->temporary)
+ vm_page_set_state(page,
PAGE_STATE_INACTIVE);
+ else if (page->modified)
+ vm_page_set_state(page,
PAGE_STATE_MODIFIED);
+ else
+ vm_page_set_state(page,
PAGE_STATE_CACHED);
+ }
+ }
+
+ if (unmapPages || cache != area->cache) {
addr_t address = area->Base()
+ ((page->cache_offset * B_PAGE_SIZE) -
area->cache_offset);
@@ -1040,6 +1076,129 @@
}
+bool
+X86VMTranslationMap::ClearAccessedAndModified(VMArea* area, addr_t address,
+ bool unmapIfUnaccessed, bool& _modified)
+{
+ ASSERT(address % B_PAGE_SIZE == 0);
+
+ page_directory_entry* pd = fArchData->pgdir_virt;
+
+ TRACE(("X86VMTranslationMap::ClearAccessedAndModified(%#" B_PRIxADDR
")\n",
+ address));
+
+ RecursiveLocker locker(fLock);
+
+ int index = VADDR_TO_PDENT(address);
+ if ((pd[index] & X86_PDE_PRESENT) == 0)
+ return false;
+
+ ThreadCPUPinner pinner(thread_get_current_thread());
+
+ page_table_entry* pt = fArchData->page_mapper->GetPageTableAt(
+ pd[index] & X86_PDE_ADDRESS_MASK);
+
+ index = VADDR_TO_PTENT(address);
+
+ // perform the deed
+ page_table_entry oldEntry;
+
+ if (unmapIfUnaccessed) {
+ while (true) {
+ oldEntry = pt[index];
+ if ((oldEntry & X86_PTE_PRESENT) == 0) {
+ // page mapping not valid
+ return false;
+ }
+
+ if (oldEntry & X86_PTE_ACCESSED) {
+ // page was accessed -- just clear the flags
+ oldEntry =
clear_page_table_entry_flags(&pt[index],
+ X86_PTE_ACCESSED | X86_PTE_DIRTY);
+ break;
+ }
+
+ // page hasn't been accessed -- unmap it
+ if (test_and_set_page_table_entry(&pt[index], 0,
oldEntry)
+ == oldEntry) {
+ break;
+ }
+
+ // something changed -- check again
+ }
+ } else {
+ oldEntry = clear_page_table_entry_flags(&pt[index],
+ X86_PTE_ACCESSED | X86_PTE_DIRTY);
+ }
+
+ pinner.Unlock();
+
+ _modified = (oldEntry & X86_PTE_DIRTY) != 0;
+
+ if ((oldEntry & X86_PTE_ACCESSED) != 0) {
+ // Note, that we only need to invalidate the address, if the
+ // accessed flags was set, since only then the entry could have
been
+ // in any TLB.
+ if (fArchData->num_invalidate_pages
+ < PAGE_INVALIDATE_CACHE_SIZE) {
+
fArchData->pages_to_invalidate[fArchData->num_invalidate_pages]
+ = address;
+ }
+
+ fArchData->num_invalidate_pages++;
+
+ Flush();
+
+ return true;
+ }
+
+ if (!unmapIfUnaccessed)
+ return false;
+
+ // We have unmapped the address. Do the "high level" stuff.
+
+ fMapCount--;
+
+ if (area->cache_type == CACHE_TYPE_DEVICE)
+ return false;
+
+ // get the page
+ vm_page* page = vm_lookup_page(
+ (oldEntry & X86_PTE_ADDRESS_MASK) / B_PAGE_SIZE);
+ ASSERT(page != NULL);
+
+ // remove the mapping object/decrement the wired_count of the page
+ vm_page_mapping* mapping = NULL;
+ if (area->wiring == B_NO_LOCK) {
+ vm_page_mappings::Iterator iterator =
page->mappings.GetIterator();
+ while ((mapping = iterator.Next()) != NULL) {
+ if (mapping->area == area) {
+ area->mappings.Remove(mapping);
+ page->mappings.Remove(mapping);
+ break;
+ }
+ }
+
+ ASSERT(mapping != NULL);
+ } else
+ page->wired_count--;
+
+ locker.Unlock();
+
+ if (page->wired_count == 0 && page->mappings.IsEmpty())
+ atomic_add(&gMappedPagesCount, -1);
+
+ if (mapping != NULL) {
+ object_cache_free(gPageMappingsObjectCache, mapping,
+ CACHE_DONT_WAIT_FOR_MEMORY |
CACHE_DONT_LOCK_KERNEL_SPACE);
+ // Since this is called by the page daemon, we never
want to lock
+ // the kernel address space.
+ }
+
+ return false;
+}
+
+
void
X86VMTranslationMap::Flush()
{
Modified: haiku/trunk/src/system/kernel/cache/file_cache.cpp
===================================================================
--- haiku/trunk/src/system/kernel/cache/file_cache.cpp 2010-02-03 18:47:07 UTC
(rev 35392)
+++ haiku/trunk/src/system/kernel/cache/file_cache.cpp 2010-02-03 18:53:52 UTC
(rev 35393)
@@ -75,7 +75,7 @@
size_t
size);
~PrecacheIO();
- status_t Prepare();
+ status_t
Prepare(vm_page_reservation* reservation);
void ReadAsync();
virtual void IOFinished(status_t status,
@@ -99,7 +99,7 @@
typedef status_t (*cache_func)(file_cache_ref* ref, void* cookie, off_t offset,
int32 pageOffset, addr_t buffer, size_t bufferSize, bool useBuffer,
- size_t lastReservedPages, size_t reservePages);
+ vm_page_reservation* reservation, size_t reservePages);
static void add_to_iovec(iovec* vecs, uint32 &index, uint32 max, addr_t
address,
size_t size);
@@ -141,7 +141,7 @@
status_t
-PrecacheIO::Prepare()
+PrecacheIO::Prepare(vm_page_reservation* reservation)
{
if (fPageCount == 0)
return B_BAD_VALUE;
@@ -157,8 +157,8 @@
// allocate pages for the cache and mark them busy
uint32 i = 0;
for (size_t pos = 0; pos < fSize; pos += B_PAGE_SIZE) {
- vm_page* page = vm_page_allocate_page(
- PAGE_STATE_ACTIVE | VM_PAGE_ALLOC_BUSY);
+ vm_page* page = vm_page_allocate_page(reservation,
+ PAGE_STATE_CACHED | VM_PAGE_ALLOC_BUSY);
fCache->InsertPage(page, fOffset + pos);
@@ -281,7 +281,8 @@
static void
-reserve_pages(file_cache_ref* ref, size_t reservePages, bool isWrite)
+reserve_pages(file_cache_ref* ref, vm_page_reservation* reservation,
+ size_t reservePages, bool isWrite)
{
if (low_resource_state(B_KERNEL_RESOURCE_PAGES) != B_NO_LOW_RESOURCE) {
VMCache* cache = ref->cache;
@@ -320,7 +321,7 @@
cache->Unlock();
}
- vm_page_reserve_pages(reservePages, VM_PRIORITY_USER);
+ vm_page_reserve_pages(reservation, reservePages, VM_PRIORITY_USER);
}
@@ -367,7 +368,7 @@
static status_t
read_into_cache(file_cache_ref* ref, void* cookie, off_t offset,
int32 pageOffset, addr_t buffer, size_t bufferSize, bool useBuffer,
- size_t lastReservedPages, size_t reservePages)
+ vm_page_reservation* reservation, size_t reservePages)
{
TRACE(("read_into_cache(offset = %Ld, pageOffset = %ld, buffer = %#lx, "
"bufferSize = %lu\n", offset, pageOffset, buffer, bufferSize));
@@ -386,7 +387,7 @@
// allocate pages for the cache and mark them busy
for (size_t pos = 0; pos < numBytes; pos += B_PAGE_SIZE) {
vm_page* page = pages[pageIndex++] = vm_page_allocate_page(
- PAGE_STATE_ACTIVE | VM_PAGE_ALLOC_BUSY);
+ reservation, PAGE_STATE_CACHED | VM_PAGE_ALLOC_BUSY);
cache->InsertPage(page, offset + pos);
@@ -397,7 +398,7 @@
push_access(ref, offset, bufferSize, false);
cache->Unlock();
- vm_page_unreserve_pages(lastReservedPages);
+ vm_page_unreserve_pages(reservation);
// read file into reserved pages
status_t status = read_pages_and_clear_partial(ref, cookie, offset,
vecs,
@@ -434,7 +435,7 @@
}
}
- reserve_pages(ref, reservePages, false);
+ reserve_pages(ref, reservation, reservePages, false);
cache->Lock();
// make the pages accessible in the cache
@@ -451,7 +452,7 @@
static status_t
read_from_file(file_cache_ref* ref, void* cookie, off_t offset,
int32 pageOffset, addr_t buffer, size_t bufferSize, bool useBuffer,
- size_t lastReservedPages, size_t reservePages)
+ vm_page_reservation* reservation, size_t reservePages)
{
TRACE(("read_from_file(offset = %Ld, pageOffset = %ld, buffer = %#lx, "
"bufferSize = %lu\n", offset, pageOffset, buffer, bufferSize));
@@ -465,13 +466,13 @@
push_access(ref, offset, bufferSize, false);
ref->cache->Unlock();
- vm_page_unreserve_pages(lastReservedPages);
+ vm_page_unreserve_pages(reservation);
status_t status = vfs_read_pages(ref->vnode, cookie, offset +
pageOffset,
&vec, 1, 0, &bufferSize);
if (status == B_OK)
- reserve_pages(ref, reservePages, false);
+ reserve_pages(ref, reservation, reservePages, false);
ref->cache->Lock();
@@ -487,7 +488,7 @@
static status_t
write_to_cache(file_cache_ref* ref, void* cookie, off_t offset,
int32 pageOffset, addr_t buffer, size_t bufferSize, bool useBuffer,
- size_t lastReservedPages, size_t reservePages)
+ vm_page_reservation* reservation, size_t reservePages)
{
// TODO: We're using way too much stack! Rather allocate a sufficiently
// large chunk on the heap.
@@ -509,7 +510,9 @@
// TODO: the pages we allocate here should have been reserved
upfront
// in cache_io()
vm_page* page = pages[pageIndex++] = vm_page_allocate_page(
- PAGE_STATE_ACTIVE | VM_PAGE_ALLOC_BUSY);
+ reservation,
+ (writeThrough ? PAGE_STATE_CACHED : PAGE_STATE_MODIFIED)
+ | VM_PAGE_ALLOC_BUSY);
ref->cache->InsertPage(page, offset + pos);
@@ -519,7 +522,7 @@
push_access(ref, offset, bufferSize, true);
ref->cache->Unlock();
- vm_page_unreserve_pages(lastReservedPages);
+ vm_page_unreserve_pages(reservation);
// copy contents (and read in partially written pages first)
@@ -601,7 +604,7 @@
}
if (status == B_OK)
- reserve_pages(ref, reservePages, true);
+ reserve_pages(ref, reservation, reservePages, true);
ref->cache->Lock();
@@ -609,9 +612,6 @@
for (int32 i = pageIndex; i-- > 0;) {
ref->cache->MarkPageUnbusy(pages[i]);
- if (!writeThrough)
- vm_page_set_state(pages[i], PAGE_STATE_MODIFIED);
-
DEBUG_PAGE_ACCESS_END(pages[i]);
}
@@ -621,12 +621,12 @@
static status_t
write_to_file(file_cache_ref* ref, void* cookie, off_t offset, int32
pageOffset,
- addr_t buffer, size_t bufferSize, bool useBuffer, size_t
lastReservedPages,
- size_t reservePages)
+ addr_t buffer, size_t bufferSize, bool useBuffer,
+ vm_page_reservation* reservation, size_t reservePages)
{
push_access(ref, offset, bufferSize, true);
ref->cache->Unlock();
- vm_page_unreserve_pages(lastReservedPages);
+ vm_page_unreserve_pages(reservation);
status_t status = B_OK;
@@ -652,7 +652,7 @@
}
if (status == B_OK)
- reserve_pages(ref, reservePages, true);
+ reserve_pages(ref, reservation, reservePages, true);
ref->cache->Lock();
@@ -665,7 +665,7 @@
off_t offset, addr_t buffer, bool useBuffer, int32 &pageOffset,
size_t bytesLeft, size_t &reservePages, off_t &lastOffset,
addr_t &lastBuffer, int32 &lastPageOffset, size_t &lastLeft,
- size_t &lastReservedPages)
+ size_t &lastReservedPages, vm_page_reservation* reservation)
{
if (lastBuffer == buffer)
return B_OK;
@@ -675,7 +675,7 @@
+ lastPageOffset + B_PAGE_SIZE - 1) >> PAGE_SHIFT);
status_t status = function(ref, cookie, lastOffset, lastPageOffset,
- lastBuffer, requestSize, useBuffer, lastReservedPages,
reservePages);
+ lastBuffer, requestSize, useBuffer, reservation, reservePages);
if (status == B_OK) {
lastReservedPages = reservePages;
lastBuffer = buffer;
@@ -736,7 +736,9 @@
size_t pagesProcessed = 0;
cache_func function = NULL;
- reserve_pages(ref, lastReservedPages, doWrite);
+ vm_page_reservation reservation;
+ reserve_pages(ref, &reservation, lastReservedPages, doWrite);
+
AutoLocker<VMCache> locker(cache);
while (bytesLeft > 0) {
@@ -763,7 +765,7 @@
status_t status = satisfy_cache_io(ref, cookie,
function, offset,
buffer, useBuffer, pageOffset, bytesLeft,
reservePages,
lastOffset, lastBuffer, lastPageOffset,
lastLeft,
- lastReservedPages);
+ lastReservedPages, &reservation);
if (status != B_OK)
return status;
@@ -779,13 +781,6 @@
"= %lu\n", offset, page, bytesLeft, pageOffset));
if (page != NULL) {
- // Since we don't actually map pages as part of an
area, we have
- // to manually maintain their usage_count
- page->usage_count = 2;
- // TODO: Just because this request comes from
the FS API, it
- // doesn't mean the page is not mapped. We
might actually
- // decrease the usage count of a hot page here.
-
if (doWrite || useBuffer) {
// Since the following user_mem{cpy,set}()
might cause a page
// fault, which in turn might cause pages to be
reserved, we
@@ -824,10 +819,19 @@
if (bytesLeft <= bytesInPage) {
// we've read the last page, so we're done!
locker.Unlock();
- vm_page_unreserve_pages(lastReservedPages);
+ vm_page_unreserve_pages(&reservation);
return B_OK;
}
+ // If it is cached only, requeue the page, so the
respective queue
+ // roughly remains LRU first sorted.
+ if (page->state == PAGE_STATE_CACHED
+ || page->state == PAGE_STATE_MODIFIED) {
+ DEBUG_PAGE_ACCESS_START(page);
+ vm_page_requeue(page, true);
+ DEBUG_PAGE_ACCESS_END(page);
+ }
+
// prepare a potential gap request
lastBuffer = buffer + bytesInPage;
lastLeft = bytesLeft - bytesInPage;
@@ -848,7 +852,7 @@
status_t status = satisfy_cache_io(ref, cookie,
function, offset,
buffer, useBuffer, pageOffset, bytesLeft,
reservePages,
lastOffset, lastBuffer, lastPageOffset,
lastLeft,
- lastReservedPages);
+ lastReservedPages, &reservation);
if (status != B_OK)
return status;
}
@@ -857,7 +861,7 @@
// fill the last remaining bytes of the request (either write or read)
return function(ref, cookie, lastOffset, lastPageOffset, lastBuffer,
- lastLeft, useBuffer, lastReservedPages, 0);
+ lastLeft, useBuffer, &reservation, 0);
}
@@ -929,6 +933,11 @@
if (offset + size > fileSize)
size = fileSize - offset;
+
+ // "offset" and "size" are always aligned to B_PAGE_SIZE,
+ offset = ROUNDDOWN(offset, B_PAGE_SIZE);
+ size = ROUNDUP(size, B_PAGE_SIZE);
+
size_t reservePages = size / B_PAGE_SIZE;
// Don't do anything if we don't have the resources left, or the cache
@@ -939,14 +948,11 @@
return;
}
- // "offset" and "size" are always aligned to B_PAGE_SIZE,
- offset &= ~(B_PAGE_SIZE - 1);
- size = ROUNDUP(size, B_PAGE_SIZE);
-
size_t bytesToRead = 0;
off_t lastOffset = offset;
- vm_page_reserve_pages(reservePages, VM_PRIORITY_USER);
+ vm_page_reservation reservation;
+ vm_page_reserve_pages(&reservation, reservePages, VM_PRIORITY_USER);
cache->Lock();
@@ -965,9 +971,9 @@
}
if (bytesToRead != 0) {
// read the part before the current page (or the end of
the request)
- PrecacheIO* io
- = new(std::nothrow) PrecacheIO(ref, lastOffset,
bytesToRead);
- if (io == NULL || io->Prepare() != B_OK) {
+ PrecacheIO* io = new(std::nothrow) PrecacheIO(ref,
lastOffset,
+ bytesToRead);
+ if (io == NULL || io->Prepare(&reservation) != B_OK) {
delete io;
break;
}
@@ -989,9 +995,7 @@
}
cache->ReleaseRefAndUnlock();
- vm_page_unreserve_pages(reservePages);
- // TODO: We should periodically unreserve as we go, so we don't
- // unnecessarily put pressure on the free page pool.
+ vm_page_unreserve_pages(&reservation);
}
@@ -1074,10 +1078,11 @@
file_cache_init(void)
{
// allocate a clean page we can use for writing zeroes
- vm_page_reserve_pages(1, VM_PRIORITY_SYSTEM);
- vm_page* page = vm_page_allocate_page(
+ vm_page_reservation reservation;
+ vm_page_reserve_pages(&reservation, 1, VM_PRIORITY_SYSTEM);
+ vm_page* page = vm_page_allocate_page(&reservation,
PAGE_STATE_WIRED | VM_PAGE_ALLOC_CLEAR);
- vm_page_unreserve_pages(1);
+ vm_page_unreserve_pages(&reservation);
sZeroPage = (addr_t)page->physical_page_number * B_PAGE_SIZE;
@@ -1319,4 +1324,3 @@
return status;
}
-
Modified: haiku/trunk/src/system/kernel/low_resource_manager.cpp
===================================================================
--- haiku/trunk/src/system/kernel/low_resource_manager.cpp 2010-02-03
18:47:07 UTC (rev 35392)
+++ haiku/trunk/src/system/kernel/low_resource_manager.cpp 2010-02-03
18:53:52 UTC (rev 35393)
@@ -301,9 +301,6 @@
switch (resource) {
case B_KERNEL_RESOURCE_PAGES:
- vm_schedule_page_scanner(requirements);
- break;
-
case B_KERNEL_RESOURCE_MEMORY:
case B_KERNEL_RESOURCE_SEMAPHORES:
case B_KERNEL_RESOURCE_ADDRESS_SPACE:
Modified: haiku/trunk/src/system/kernel/slab/MemoryManager.cpp
===================================================================
--- haiku/trunk/src/system/kernel/slab/MemoryManager.cpp 2010-02-03
18:47:07 UTC (rev 35392)
+++ haiku/trunk/src/system/kernel/slab/MemoryManager.cpp 2010-02-03
18:53:52 UTC (rev 35393)
@@ -1333,13 +1333,14 @@
// reserve the pages we need now
size_t reservedPages = size / B_PAGE_SIZE
+ translationMap->MaxPagesNeededToMap(address, address + size -
1);
+ vm_page_reservation reservation;
if ((flags & CACHE_DONT_WAIT_FOR_MEMORY) != 0) {
- if (!vm_page_try_reserve_pages(reservedPages, priority)) {
+ if (!vm_page_try_reserve_pages(&reservation, reservedPages,
priority)) {
vm_unreserve_memory(reservedMemory);
return B_WOULD_BLOCK;
}
} else
- vm_page_reserve_pages(reservedPages, priority);
+ vm_page_reserve_pages(&reservation, reservedPages, priority);
VMCache* cache = vm_area_get_locked_cache(vmArea);
@@ -1350,7 +1351,7 @@
addr_t endAreaOffset = areaOffset + size;
for (size_t offset = areaOffset; offset < endAreaOffset;
offset += B_PAGE_SIZE) {
- vm_page* page = vm_page_allocate_page(PAGE_STATE_WIRED);
+ vm_page* page = vm_page_allocate_page(&reservation,
PAGE_STATE_WIRED);
cache->InsertPage(page, offset);
page->wired_count++;
@@ -1359,14 +1360,14 @@
translationMap->Map(vmArea->Base() + offset,
page->physical_page_number * B_PAGE_SIZE,
- B_KERNEL_READ_AREA | B_KERNEL_WRITE_AREA);
+ B_KERNEL_READ_AREA | B_KERNEL_WRITE_AREA, &reservation);
}
translationMap->Unlock();
[... truncated: 2593 lines follow ...]
Other related posts:
- » [haiku-commits] r35393 - in haiku/trunk: headers/private/kernel/vm src/add-ons/kernel/bus_managers/agp_gart src/system/kernel src/system/kernel/arch/ppc src/system/kernel/arch/x86 ... - ingo_weinhold
